-----============= acceptance-small: sanityn ============----- Sun Nov 24 01:11:07 EST 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 01:11:11 (1732428671) === Starting client oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Started clients oleg356-client.virtnet: 192.168.203.156@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg356-client.virtnet: executing check_config_client /mnt/lustre oleg356-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg356-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012a4fe800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012d0d7800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a4fe800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012d0d7800.idle_timeout=debug disable quota as required oleg356-server: oleg356-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 01:11:19 (1732428679) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133577 s, 78.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6978] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 01:11:20 (1732428680) PASS 0 (2s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 01:11:22 (1732428682) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 01:11:23 (1732428683) -rw-r--r-- 1 root root 0 Nov 24 01:11 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 01:11:25 (1732428685) -rw-r--r-- 1 root root 0 Nov 24 01:11 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 01:11:26 (1732428686) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 01:11:28 (1732428688) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 01:11:29 (1732428689) total 1 -rwxrwxrwx 1 root root 0 Nov 24 01:11 f2a -rwxrwxrwx 1 root root 0 Nov 24 01:11 f2b total 1 -rwxrwxrwx 1 root root 0 Nov 24 01:11 f2a -rwxrwxrwx 1 root root 0 Nov 24 01:11 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00259758 s, 197 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 01:11:30 (1732428690) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 01:11:31 (1732428691) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0672615 s, 31.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 01:11:32 (1732428692) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 01:11:34 (1732428694) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 01:11:36 (1732428696) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 01:11:37 (1732428697) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 01:11:39 (1732428699) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) == sanityn test 8: remove of open special file on other node ==================================================================== 01:11:40 (1732428700) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 01:11:42 (1732428702) PASS 9a (1s) == sanityn test 9b: append to striped sparse file ======== 01:11:43 (1732428703) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108399 s, 96.7 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.00708789 s, 0.4 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 01:11:45 (1732428705) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00112273 s, 0.9 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00203068 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00235066 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00366375 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00219363 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00238151 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00211007 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00291336 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0024749 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00208799 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00313162 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00238768 s, 0.4 kB/s PASS 10a (1s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 01:11:46 (1732428706) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00228836 s, 1.3 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00572867 s, 715 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000195465 s, 15.7 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 01:11:48 (1732428708) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (1s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 01:11:49 (1732428709) warning: '-runtime' deprecated, use '-t runtime' instead - link 3149 (time 1732428720.08 total 10.00 last 314.81) - link 6286 (time 1732428730.08 total 20.00 last 313.68) - link 9762 (time 1732428740.08 total 30.00 last 347.58) - link 13259 (time 1732428750.08 total 40.01 last 349.66) - link 16572 (time 1732428760.09 total 50.01 last 331.23) - link 19885 (time 1732428770.09 total 60.01 last 331.25) - link 23005 (time 1732428780.09 total 70.01 last 311.97) - link 26408 (time 1732428790.09 total 80.01 last 340.28) - link 29658 (time 1732428800.09 total 90.01 last 324.91) total: 33179 link in 100.00 seconds: 331.79 ops/second using seed 1684584822 running for 100 seconds - stat 10000 (time 1732428716 ; total 5 ; last 5) - stat 20000 (time 1732428721 ; total 10 ; last 5) - stat 30000 (time 1732428725 ; total 14 ; last 4) - stat 40000 (time 1732428730 ; total 19 ; last 5) - stat 50000 (time 1732428735 ; total 24 ; last 5) - stat 60000 (time 1732428739 ; total 28 ; last 4) - stat 70000 (time 1732428744 ; total 33 ; last 5) - stat 80000 (time 1732428748 ; total 37 ; last 4) - stat 90000 (time 1732428753 ; total 42 ; last 5) - stat 100000 (time 1732428758 ; total 47 ; last 5) - stat 110000 (time 1732428766 ; total 55 ; last 8) - stat 118704 (time 1732428777 ; total 66 ; last 11) - stat 120000 (time 1732428778 ; total 67 ; last 1) - stat 128744 (time 1732428789 ; total 78 ; last 11) - stat 130000 (time 1732428790 ; total 79 ; last 1) - stat 138908 (time 1732428801 ; total 90 ; last 11) - stat 140000 (time 1732428802 ; total 91 ; last 1) total: 149508 stats in 100 seconds: 1495.079956 stats/second - unlinked 0 (time 1732428761 ; total 0 ; last 0) - unlinked 10000 (time 1732428808 ; total 47 ; last 47) - unlinked 20000 (time 1732428830 ; total 69 ; last 22) - unlinked 30000 (time 1732428854 ; total 93 ; last 24) unlink(/mnt/lustre2/lockdir/lockfile33179) error: No such file or directory total: 33178 unlinks in 100 seconds: 331.779999 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21828) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21831) - No such process PASS 12 (153s) == sanityn test 13: test directory page revocation ======= 01:14:22 (1732428862) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 01:14:24 (1732428864) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4851: 23226 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 01:14:25 (1732428865) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4851: 23860 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 01:14:27 (1732428867) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24492 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 01:14:28 (1732428868) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25126 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 01:14:29 (1732428869) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 25761 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 01:14:31 (1732428871) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 01:14:43 (1732428883) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.112057 s, 74.9 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.107038 s, 78.4 MB/s Chance of close/open is 1 in 50 Seed set to 8884 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1732428884.309520 mapwrite 0x55e8ae thru 0x567eb0 (0x9603 bytes) 000006[0] 1732428884.340159 mapwrite 0x7a27e7 thru 0x7a9fd4 (0x77ee bytes) truncating to largest ever: 0x332164 000008[0] 1732428884.367549 trunc from 0x7a9fd5 to 0x332163 (0x477e71 bytes) 000011[1] 1732428884.380191 write 0x9886ab thru 0x993f6a (0xb8c0 bytes) 000012[0] 1732428884.387374 write 0x5ecaef thru 0x5f9520 (0xca32 bytes) 000013[1] 1732428884.391550 mapwrite 0x4864fc thru 0x489669 (0x316e bytes) 000014[0] 1732428884.412544 mapwrite 0x501636 thru 0x5090d8 (0x7aa3 bytes) 000015[1] 1732428884.435920 mapread 0x240927 thru 0x24a7a4 (0x9e7e bytes) truncating to largest ever: 0x90bb11 000018[0] 1732428884.451611 trunc from 0x993f6b to 0x90bb10 (0x8845a bytes) 000019[1] 1732428884.462395 read 0x67de79 thru 0x684be0 (0x6d68 bytes) 000020[0] 1732428884.469124 mapread 0x1e5f92 thru 0x1ea2df (0x434e bytes) 000021[1] 1732428884.477210 trunc from 0x90bb11 to 0x590cdc (0x37ae34 bytes) 000024[1] 1732428884.488138 write 0x2cd0c6 thru 0x2db17e (0xe0b9 bytes) 000028[1] 1732428884.494620 write 0x2af523 thru 0x2b00b1 (0xb8f bytes) 000029[1] 1732428884.500232 mapread 0x1cd211 thru 0x1dd027 (0xfe17 bytes) 000033[0] 1732428884.509182 write 0x61debe thru 0x6233c1 (0x5504 bytes) 000039[0] 1732428884.530906 write 0x711c90 thru 0x717b02 (0x5e73 bytes) 000040[0] 1732428884.534684 write 0x425915 thru 0x42e940 (0x902c bytes) 000042[0] 1732428884.542983 mapwrite 0x2b7039 thru 0x2c3417 (0xc3df bytes) 000044[1] 1732428884.573827 read 0x45d0c9 thru 0x46599d (0x88d5 bytes) 000045[0] 1732428884.588779 mapwrite 0x9f2c9c thru 0x9f2d06 (0x6b bytes) 000046[0] 1732428884.607213 mapread 0x2b1e72 thru 0x2bdedc (0xc06b bytes) 000047[1] 1732428884.612908 read 0x31d99d thru 0x324644 (0x6ca8 bytes) 000048[0] 1732428884.634125 trunc from 0x9f2d07 to 0x615fe0 (0x3dcd26 bytes) 000052[0] 1732428884.647480 trunc from 0x615fe1 to 0x7f3d93 (0x1dddb3 bytes) 000053[0] 1732428884.651720 write 0x4591be thru 0x4618a9 (0x86ec bytes) 000054[1] 1732428884.654305 mapread 0x15ef1b thru 0x16a1fc (0xb2e2 bytes) 000055[1] 1732428884.669881 mapwrite 0x3bf896 thru 0x3cf5af (0xfd1a bytes) 000059[1] 1732428884.717247 write 0x727bf7 thru 0x72cfa4 (0x53ae bytes) 000061[1] 1732428884.744942 trunc from 0x7f3d94 to 0x3cad0f (0x429084 bytes) 000069[1] 1732428884.768711 read 0x31d163 thru 0x31f3bb (0x2259 bytes) 000072[1] 1732428884.797949 read 0x10397e thru 0x10ceb0 (0x9533 bytes) 000073[1] 1732428884.798277 mapread 0x10a513 thru 0x11794b (0xd439 bytes) 000081[1] 1732428884.815425 mapread 0x313741 thru 0x3218e3 (0xe1a3 bytes) 000082[1] 1732428884.819092 mapwrite 0x074310 thru 0x07e4c5 (0xa1b6 bytes) 000083[0] 1732428884.833090 trunc from 0x3cad10 to 0x8c849d (0x4fd78e bytes) 000089[0] 1732428884.852990 write 0x17b98d thru 0x1804ea (0x4b5e bytes) 000091[1] 1732428884.862169 trunc from 0x8c849e to 0x3a9938 (0x51eb65 bytes) 000092[0] 1732428884.871120 write 0x16f805 thru 0x174ff1 (0x57ed bytes) 000094[1] 1732428884.881422 read 0x0b611b thru 0x0ba45f (0x4345 bytes) 000095[0] 1732428884.905076 trunc from 0x3a9939 to 0x7d4657 (0x42ad1f bytes) truncating to largest ever: 0x9d37f1 000097[1] 1732428884.923830 trunc from 0x7d4658 to 0x9d37f0 (0x1ff199 bytes) 000099[1] 1732428884.937264 write 0x827374 thru 0x82ba90 (0x471d bytes) 000101[0] 1732428884.951186 write 0x60738f thru 0x60ab13 (0x3785 bytes) 000102[0] 1732428884.968495 read 0x1b05f9 thru 0x1b2592 (0x1f9a bytes) 000104[1] 1732428884.974511 trunc from 0x9d37f1 to 0x305bbd (0x6cdc33 bytes) 000105[0] 1732428884.986003 read 0x3049ff thru 0x305bbd (0x11bf bytes) 000107[1] 1732428885.004761 trunc from 0x305bbe to 0x950dba (0x64b1fd bytes) 000111[1] 1732428885.018440 mapwrite 0x3d65b6 thru 0x3e2d9b (0xc7e6 bytes) 000120[1] 1732428885.074021 mapwrite 0x8552df thru 0x85a3ea (0x510c bytes) 000122[1] 1732428885.091377 trunc from 0x950dbb to 0x537cf2 (0x4190c8 bytes) 000124[0] 1732428885.096581 write 0x3adfac thru 0x3b6039 (0x808e bytes) 000125[1] 1732428885.109662 mapread 0x37b6c8 thru 0x389f29 (0xe862 bytes) 000126[1] 1732428885.131117 read 0x16c13d thru 0x177078 (0xaf3c bytes) 000129[1] 1732428885.158135 write 0x2c9e95 thru 0x2ceb58 (0x4cc4 bytes) 000130[1] 1732428885.168149 mapwrite 0x611723 thru 0x619393 (0x7c71 bytes) 000136[1] 1732428885.203447 mapwrite 0x8adde6 thru 0x8ba2ec (0xc507 bytes) 000137[1] 1732428885.237779 trunc from 0x8ba2ed to 0x99d9ed (0xe3701 bytes) 000141[1] 1732428885.252690 mapwrite 0x96a4e8 thru 0x96bcf5 (0x180e bytes) 000142[0] 1732428885.269009 mapwrite 0x0cad47 thru 0x0d813c (0xd3f6 bytes) 000148[1] 1732428885.322104 mapwrite 0x6d4594 thru 0x6e310c (0xeb79 bytes) 000150[0] 1732428885.341278 mapread 0x1310a0 thru 0x13860e (0x756f bytes) 000153[0] 1732428885.360797 mapwrite 0x43d721 thru 0x44c979 (0xf259 bytes) 000154[1] 1732428885.398880 mapread 0x318bf2 thru 0x32606c (0xd47b bytes) truncating to largest ever: 0x9e15d6 000156[1] 1732428885.420954 trunc from 0x99d9ee to 0x9e15d5 (0x43be8 bytes) 000159[0] 1732428885.436409 mapread 0x6ddb03 thru 0x6ebb1d (0xe01b bytes) 000161[0] 1732428885.456892 read 0x7348a2 thru 0x73db2d (0x928c bytes) 000167[0] 1732428885.476816 trunc from 0x9e15d6 to 0x0a2e09 (0x93e7cc bytes) 000169[0] 1732428885.490894 write 0x9d8240 thru 0x9e5fbc (0xdd7d bytes) 000170[1] 1732428885.496538 read 0x534c0e thru 0x53a98d (0x5d80 bytes) 000175[1] 1732428885.511922 write 0x34c24e thru 0x352a14 (0x67c7 bytes) 000176[0] 1732428885.543643 mapwrite 0x182008 thru 0x1865f9 (0x45f2 bytes) 000177[1] 1732428885.588211 mapwrite 0x921d7a thru 0x92a9e7 (0x8c6e bytes) 000178[1] 1732428885.628994 mapwrite 0x1ffb43 thru 0x20f159 (0xf617 bytes) 000179[0] 1732428885.664115 read 0x4f2e4d thru 0x4fd70c (0xa8c0 bytes) 000186[1] 1732428885.689322 read 0x279180 thru 0x27f9ca (0x684b bytes) 000188[1] 1732428885.717918 mapread 0x39a08e thru 0x3a4cca (0xac3d bytes) 000189[0] 1732428885.718409 mapwrite 0x84ef8d thru 0x84f51b (0x58f bytes) 000194[0] 1732428885.768535 mapwrite 0x952e5c thru 0x95e77f (0xb924 bytes) truncating to largest ever: 0x9f9d90 000196[0] 1732428885.799879 trunc from 0x9e5fbd to 0x9f9d8f (0x13dd3 bytes) 000197[1] 1732428885.809823 mapwrite 0x409995 thru 0x40c731 (0x2d9d bytes) 000198[1] 1732428885.831439 trunc from 0x9f9d90 to 0x3079b9 (0x6f23d6 bytes) 000199[1] 1732428885.846977 read 0x016a46 thru 0x019f62 (0x351d bytes) 000200[0] 1732428885.863577 read 0x01462a thru 0x020eff (0xc8d6 bytes) 000201[0] 1732428885.884738 mapwrite 0x3a02cf thru 0x3a4fff (0x4d31 bytes) 000203[0] 1732428885.919445 mapread 0x374756 thru 0x37b9ed (0x7298 bytes) 000209[1] 1732428885.927787 write 0x63bb4c thru 0x6443cf (0x8884 bytes) 000212[0] 1732428885.940489 mapwrite 0x52fa7c thru 0x53d49d (0xda22 bytes) 000214[0] 1732428885.979985 mapwrite 0x0c8ac3 thru 0x0d2b66 (0xa0a4 bytes) 000215[0] 1732428886.004644 trunc from 0x6443d0 to 0x066fe4 (0x5dd3eb bytes) 000216[0] 1732428886.009947 read 0x001cd8 thru 0x007c2b (0x5f54 bytes) 000218[1] 1732428886.019263 write 0x311f65 thru 0x3189d5 (0x6a71 bytes) 000219[1] 1732428886.026434 read 0x0c53c4 thru 0x0cd743 (0x8380 bytes) 000222[0] 1732428886.051090 read 0x1c1387 thru 0x1cf896 (0xe510 bytes) 000223[1] 1732428886.075424 write 0x793ce6 thru 0x7a0482 (0xc79d bytes) 000224[0] 1732428886.082603 read 0x03767a thru 0x039ea7 (0x282e bytes) 000226[1] 1732428886.089591 read 0x6ab36d thru 0x6b31c7 (0x7e5b bytes) 000227[0] 1732428886.101095 read 0x10fabb thru 0x11366c (0x3bb2 bytes) 000229[0] 1732428886.105329 write 0x9f803d thru 0x9f8cf6 (0xcba bytes) 000237[1] 1732428886.133998 trunc from 0x9f8cf7 to 0x9192f7 (0xdf9ff bytes) 000238[0] 1732428886.140489 write 0x2265dd thru 0x2354bd (0xeee1 bytes) 000242[1] 1732428886.178100 mapwrite 0x405a13 thru 0x405bdc (0x1ca bytes) 000243[1] 1732428886.195967 mapwrite 0x612395 thru 0x619717 (0x7383 bytes) 000246[0] 1732428886.226926 trunc from 0x9192f8 to 0x2970dd (0x68221a bytes) 000248[0] 1732428886.243309 mapwrite 0x336ef7 thru 0x33b2c8 (0x43d2 bytes) 000250[1] 1732428886.269433 trunc from 0x33b2c9 to 0x7dc45f (0x4a1197 bytes) 000251[1] 1732428886.280815 trunc from 0x7dc460 to 0x6d8ae7 (0x103978 bytes) 000252[1] 1732428886.287281 mapread 0x4c3414 thru 0x4cbbf6 (0x87e3 bytes) 000253[1] 1732428886.313642 write 0x9eed38 thru 0x9f1f8d (0x3256 bytes) 000254[1] 1732428886.316643 mapread 0x29a907 thru 0x2a9a32 (0xf12c bytes) 000260[0] 1732428886.325904 write 0x872cc3 thru 0x87f9ac (0xccea bytes) 000261[1] 1732428886.339769 trunc from 0x9f1f8e to 0x5e5a91 (0x40c4fc bytes) 000262[0] 1732428886.349774 trunc from 0x5e5a92 to 0x235ddb (0x3afcb6 bytes) 000268[1] 1732428886.372410 read 0x037bfa thru 0x047188 (0xf58f bytes) 000269[0] 1732428886.388573 mapwrite 0x042b77 thru 0x0483bd (0x5847 bytes) 000270[1] 1732428886.415328 trunc from 0x235ddc to 0x7f4651 (0x5be876 bytes) 000271[1] 1732428886.424113 mapread 0x56b64e thru 0x5705b2 (0x4f65 bytes) 000275[1] 1732428886.455334 trunc from 0x7f4652 to 0x48fda6 (0x3648ab bytes) 000283[0] 1732428886.468432 write 0x3d3430 thru 0x3e1a56 (0xe627 bytes) 000284[1] 1732428886.477782 read 0x280c7c thru 0x284826 (0x3bab bytes) 000285[0] 1732428886.501925 read 0x3698cb thru 0x36c04d (0x2783 bytes) 000288[1] 1732428886.526216 read 0x0efeb6 thru 0x0f25e8 (0x2733 bytes) 000290[0] 1732428886.532615 mapwrite 0x02b905 thru 0x03b24f (0xf94b bytes) 000292[0] 1732428886.589838 mapwrite 0x1ce80a thru 0x1ce955 (0x14c bytes) 000293[1] 1732428886.606987 read 0x2afd11 thru 0x2b62df (0x65cf bytes) 000295[0] 1732428886.632513 trunc from 0x48fda7 to 0x15d2ce (0x332ad8 bytes) 000297[0] 1732428886.650212 write 0x0bc815 thru 0x0c68dc (0xa0c8 bytes) 000298[1] 1732428886.653010 mapwrite 0x9ff963 thru 0x9fffff (0x69d bytes) 000299[1] 1732428886.683581 mapwrite 0x23ff00 thru 0x24fc37 (0xfd38 bytes) 000301[1] 1732428886.729167 trunc from 0xa00000 to 0x6aa03e (0x355fc1 bytes) 000302[1] 1732428886.739842 trunc from 0x6aa03f to 0x27405c (0x435fe2 bytes) 000303[0] 1732428886.745865 read 0x196a35 thru 0x1978ff (0xecb bytes) 000304[1] 1732428886.762020 read 0x0417cc thru 0x04961d (0x7e52 bytes) 000305[1] 1732428886.778608 mapwrite 0x25df31 thru 0x26ba3b (0xdb0b bytes) 000306[1] 1732428886.822919 mapwrite 0x7c803a thru 0x7cbebe (0x3e85 bytes) 000307[1] 1732428886.861103 read 0x70d8c0 thru 0x716f7d (0x96be bytes) 000309[1] 1732428886.867400 trunc from 0x7cbebf to 0x7f3bb2 (0x27cf4 bytes) 000311[0] 1732428886.875732 trunc from 0x7f3bb3 to 0x06d05b (0x786b57 bytes) 000312[0] 1732428886.891352 mapread 0x00e014 thru 0x01c55f (0xe54c bytes) 000319[0] 1732428886.920782 mapwrite 0x766410 thru 0x775c27 (0xf818 bytes) 000320[1] 1732428886.964094 write 0x436f2b thru 0x439606 (0x26dc bytes) 000321[1] 1732428886.974016 read 0x65c5f4 thru 0x66affe (0xea0b bytes) 000327[0] 1732428887.008849 read 0x5a229c thru 0x5a6f0a (0x4c6f bytes) 000328[0] 1732428887.034740 trunc from 0x775c28 to 0x495067 (0x2e0bc0 bytes) 000329[1] 1732428887.043788 write 0x9545b7 thru 0x956d14 (0x275e bytes) 000334[1] 1732428887.068015 read 0x80fd2c thru 0x817bd0 (0x7ea5 bytes) 000335[1] 1732428887.095451 write 0x66bbfa thru 0x672a43 (0x6e4a bytes) 000336[1] 1732428887.106078 trunc from 0x956d15 to 0x0815be (0x8d5756 bytes) 000338[1] 1732428887.117218 write 0x797339 thru 0x79d2e0 (0x5fa8 bytes) 000340[1] 1732428887.125687 trunc from 0x79d2e1 to 0x8bc51f (0x11f23f bytes) 000341[0] 1732428887.129895 trunc from 0x8bc520 to 0x5d29c8 (0x2e9b57 bytes) 000342[1] 1732428887.138774 mapwrite 0x792e0e thru 0x796ea4 (0x4097 bytes) 000343[0] 1732428887.175955 mapread 0x3d6a93 thru 0x3db9de (0x4f4c bytes) 000345[1] 1732428887.190090 mapread 0x20bfa7 thru 0x20f865 (0x38bf bytes) 000346[0] 1732428887.198381 read 0x5ba6b3 thru 0x5c404f (0x999d bytes) 000348[0] 1732428887.229881 mapwrite 0x836137 thru 0x8438be (0xd788 bytes) 000349[0] 1732428887.262815 mapwrite 0x5fe927 thru 0x608c65 (0xa33f bytes) 000351[1] 1732428887.289579 mapwrite 0x8e85e4 thru 0x8f51e1 (0xcbfe bytes) 000352[0] 1732428887.318880 mapwrite 0x1b3952 thru 0x1c13ac (0xda5b bytes) 000353[0] 1732428887.361304 mapread 0x70b693 thru 0x70f4d7 (0x3e45 bytes) 000354[0] 1732428887.371551 trunc from 0x8f51e2 to 0x9df769 (0xea588 bytes) 000356[0] 1732428887.377586 trunc from 0x9df76a to 0x43475f (0x5ab00a bytes) 000358[0] 1732428887.381654 mapwrite 0x8161e8 thru 0x8213ae (0xb1c7 bytes) 000359[1] 1732428887.410820 read 0x6d5928 thru 0x6e2b5f (0xd238 bytes) 000361[1] 1732428887.430255 mapread 0x3d2988 thru 0x3d3c8b (0x1304 bytes) 000362[1] 1732428887.454499 mapwrite 0x79ff72 thru 0x7aeaa9 (0xeb38 bytes) 000363[0] 1732428887.490416 read 0x5f9089 thru 0x608af3 (0xfa6b bytes) 000364[1] 1732428887.521894 write 0x91e773 thru 0x92e6d3 (0xff61 bytes) 000365[0] 1732428887.531148 mapread 0x4ccd5d thru 0x4cdec0 (0x1164 bytes) 000366[0] 1732428887.536291 trunc from 0x92e6d4 to 0x562b79 (0x3cbb5a bytes) 000367[1] 1732428887.549355 mapwrite 0x918039 thru 0x924078 (0xc040 bytes) 000368[1] 1732428887.578424 mapread 0x1d15a1 thru 0x1db3bc (0x9e1c bytes) 000370[0] 1732428887.594450 trunc from 0x924079 to 0x8dfd78 (0x44300 bytes) 000373[1] 1732428887.610350 read 0x698ed1 thru 0x69fafd (0x6c2d bytes) 000375[1] 1732428887.623216 mapwrite 0x678bfb thru 0x67b2c5 (0x26cb bytes) 000376[0] 1732428887.642115 write 0x26a07e thru 0x271ddf (0x7d62 bytes) 000379[0] 1732428887.658437 read 0x6d8768 thru 0x6db242 (0x2adb bytes) 000381[0] 1732428887.687375 write 0x704444 thru 0x70ee90 (0xaa4d bytes) 000382[1] 1732428887.697454 read 0x659115 thru 0x65da18 (0x4904 bytes) 000383[0] 1732428887.706529 trunc from 0x8dfd79 to 0x985271 (0xa54f9 bytes) 000386[0] 1732428887.717795 mapread 0x93f1e6 thru 0x941215 (0x2030 bytes) 000390[0] 1732428887.732679 mapread 0x0d9616 thru 0x0dc71a (0x3105 bytes) 000391[1] 1732428887.736290 write 0x016104 thru 0x01c434 (0x6331 bytes) 000392[1] 1732428887.766925 read 0x0214b6 thru 0x029ad4 (0x861f bytes) 000393[0] 1732428887.770091 write 0x1dc611 thru 0x1ea63d (0xe02d bytes) 000395[0] 1732428887.786309 mapread 0x2422bd thru 0x247469 (0x51ad bytes) 000396[1] 1732428887.792632 read 0x4e2607 thru 0x4ede80 (0xb87a bytes) 000399[0] 1732428887.809180 mapread 0x7966e9 thru 0x7a2251 (0xbb69 bytes) 000400[1] 1732428887.822507 mapread 0x38b4ea thru 0x396a2a (0xb541 bytes) 000403[1] 1732428887.850253 read 0x22211d thru 0x22fae1 (0xd9c5 bytes) 000404[1] 1732428887.850543 mapwrite 0x22a639 thru 0x23a2d7 (0xfc9f bytes) 000408[1] 1732428887.908004 read 0x36f436 thru 0x371786 (0x2351 bytes) 000411[1] 1732428887.914935 write 0x2b6910 thru 0x2c59d6 (0xf0c7 bytes) 000413[0] 1732428887.921923 trunc from 0x985272 to 0x0ca79b (0x8baad6 bytes) 000414[1] 1732428887.934775 mapwrite 0x8a566d thru 0x8a8dd0 (0x3764 bytes) 000415[1] 1732428887.988409 mapwrite 0x420035 thru 0x4215a1 (0x156d bytes) 000416[1] 1732428888.002014 trunc from 0x8a8dd1 to 0x943e15 (0x9b045 bytes) 000417[1] 1732428888.006235 read 0x0b41e8 thru 0x0b838c (0x41a5 bytes) 000418[1] 1732428888.006483 mapwrite 0x602ee2 thru 0x60b30a (0x8429 bytes) 000419[1] 1732428888.019663 trunc from 0x943e16 to 0x537a6c (0x40c3a9 bytes) 000420[1] 1732428888.026657 read 0x3de70b thru 0x3e36ce (0x4fc4 bytes) 000421[1] 1732428888.026940 mapread 0x4cc842 thru 0x4cd57a (0xd39 bytes) 000422[1] 1732428888.027348 write 0x7dd2a1 thru 0x7df41b (0x217b bytes) 000423[1] 1732428888.028362 mapwrite 0x5f51a8 thru 0x5faf55 (0x5dae bytes) 000425[0] 1732428888.060459 trunc from 0x7df41c to 0x298867 (0x546bb4 bytes) 000431[0] 1732428888.086328 mapwrite 0x930840 thru 0x930b59 (0x31a bytes) 000435[1] 1732428888.113112 trunc from 0x930b5a to 0x0fc0f5 (0x834a64 bytes) 000436[0] 1732428888.124444 mapwrite 0x69d7a4 thru 0x69ebb9 (0x1416 bytes) 000437[0] 1732428888.144910 mapwrite 0x45ccf9 thru 0x461679 (0x4981 bytes) 000438[1] 1732428888.163482 read 0x187621 thru 0x18fdce (0x87ae bytes) 000440[0] 1732428888.192754 write 0x1d9cc5 thru 0x1deb8e (0x4eca bytes) 000442[1] 1732428888.207590 write 0x40fb3b thru 0x41a37b (0xa841 bytes) 000443[1] 1732428888.227335 mapwrite 0x34a632 thru 0x3535f3 (0x8fc2 bytes) 000445[0] 1732428888.265249 mapwrite 0x4ddc83 thru 0x4eab7d (0xcefb bytes) 000446[1] 1732428888.300834 write 0x64b936 thru 0x650de3 (0x54ae bytes) 000447[1] 1732428888.308470 write 0x00e017 thru 0x01dcf3 (0xfcdd bytes) 000449[1] 1732428888.314534 read 0x1e2354 thru 0x1f197d (0xf62a bytes) 000450[1] 1732428888.336194 write 0x972409 thru 0x97a265 (0x7e5d bytes) 000451[1] 1732428888.343328 write 0x904a81 thru 0x90e2a3 (0x9823 bytes) 000454[0] 1732428888.350014 trunc from 0x97a266 to 0x725e82 (0x2543e3 bytes) 000455[1] 1732428888.369522 read 0x1e5c9b thru 0x1f2f73 (0xd2d9 bytes) 000460[0] 1732428888.403170 write 0x5b8cc6 thru 0x5c0bf3 (0x7f2e bytes) 000466[0] 1732428888.416704 write 0x186a08 thru 0x186da7 (0x3a0 bytes) 000467[0] 1732428888.423561 read 0x1a06b8 thru 0x1a1280 (0xbc9 bytes) 000468[0] 1732428888.446699 mapread 0x20e194 thru 0x214ff1 (0x6e5e bytes) 000470[0] 1732428888.450700 mapwrite 0x02707c thru 0x0349e2 (0xd967 bytes) 000471[0] 1732428888.468137 write 0x6217b3 thru 0x62847c (0x6cca bytes) 000480[0] 1732428888.476812 read 0x18aff2 thru 0x198a76 (0xda85 bytes) 000483[1] 1732428888.483906 trunc from 0x725e83 to 0x105a60 (0x620422 bytes) 000485[1] 1732428888.505661 trunc from 0x105a61 to 0x362a73 (0x25d013 bytes) 000487[1] 1732428888.515019 mapwrite 0x7b0f99 thru 0x7b159a (0x602 bytes) 000495[0] 1732428888.564166 mapread 0x5b1f9d thru 0x5b989d (0x7901 bytes) 000496[1] 1732428888.582561 trunc from 0x7b159b to 0x44656a (0x36b030 bytes) 000497[1] 1732428888.588765 read 0x0f938e thru 0x1084eb (0xf15e bytes) 000498[0] 1732428888.609201 trunc from 0x44656b to 0x413e0a (0x32760 bytes) 000499[0] 1732428888.623778 mapread 0x0b6679 thru 0x0bb048 (0x49d0 bytes) 000500[1] 1732428888.640777 read 0x3a29f1 thru 0x3a32a0 (0x8b0 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 8888 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x5407aa 000001[0] 1732428888.895118 trunc from 00000000 to 0x5407a9 (0x5407aa bytes) 000004[0] 1732428888.909411 mapread 0x2a6000 thru 0x2adaed (0x7aee bytes) 000005[1] 1732428888.914243 mapwrite 0x41d000 thru 0x4284d5 (0xb4d6 bytes) 000006[0] 1732428888.954822 mapread 0x130000 thru 0x1365b8 (0x65b9 bytes) 000012[1] 1732428888.965255 trunc from 0x5407aa to 0x2a62ce (0x29a4db bytes) 000014[0] 1732428888.975980 write_OD 0x67e000 thru 0x68afff (0xd000 bytes) truncating to largest ever: 0x9273d0 000016[1] 1732428888.990529 trunc from 0x68b000 to 0x9273cf (0x29c3d0 bytes) 000018[0] 1732428889.003690 mapread 0x1a3000 thru 0x1ae8c8 (0xb8c9 bytes) 000021[0] 1732428889.013695 mapwrite 0x276000 thru 0x27878d (0x278e bytes) 000022[1] 1732428889.037452 mapread 0x4dc000 thru 0x4de3df (0x23e0 bytes) 000023[1] 1732428889.043809 mapwrite 0x692000 thru 0x69fd76 (0xdd77 bytes) 000024[0] 1732428889.066636 mapwrite 0x554000 thru 0x559f16 (0x5f17 bytes) 000025[0] 1732428889.085478 trunc from 0x9273d0 to 0x3635e0 (0x5c3def bytes) 000026[0] 1732428889.093664 read_OD 0x338000 thru 0x344fff (0xd000 bytes) 000029[0] 1732428889.100182 write_OD 0x67f000 thru 0x686fff (0x8000 bytes) 000030[0] 1732428889.114545 trunc from 0x687000 to 0x3b1d28 (0x2d52d7 bytes) 000031[1] 1732428889.123802 mapwrite 0x679000 thru 0x67e286 (0x5287 bytes) 000032[0] 1732428889.147861 mapread 0x486000 thru 0x48664c (0x64d bytes) 000033[1] 1732428889.155165 mapread 0x067000 thru 0x06d769 (0x676a bytes) 000035[0] 1732428889.160229 mapread 0x565000 thru 0x56edef (0x9df0 bytes) 000039[1] 1732428889.165742 mapread 0x574000 thru 0x581f7a (0xdf7b bytes) 000042[0] 1732428889.172208 read_OD 0x2f2000 thru 0x2f4fff (0x3000 bytes) 000044[0] 1732428889.174289 trunc from 0x67e287 to 0x12b9f2 (0x552894 bytes) 000045[0] 1732428889.184669 write_OD 0x29a000 thru 0x29bfff (0x2000 bytes) 000046[0] 1732428889.199142 write_OD 0x40f000 thru 0x419fff (0xb000 bytes) 000047[1] 1732428889.213259 mapwrite 0x13d000 thru 0x141456 (0x4457 bytes) 000049[1] 1732428889.232137 trunc from 0x41a000 to 0x27104a (0x1a8fb5 bytes) 000050[1] 1732428889.239846 read_OD 0x257000 thru 0x25dfff (0x7000 bytes) 000051[1] 1732428889.243767 write_OD 0x8c7000 thru 0x8c7fff (0x1000 bytes) 000055[1] 1732428889.257797 read_OD 0x49c000 thru 0x49efff (0x3000 bytes) 000056[1] 1732428889.259376 mapwrite 0x9a6000 thru 0x9b1621 (0xb622 bytes) 000057[0] 1732428889.320006 read_OD 0x79d000 thru 0x7a8fff (0xc000 bytes) 000066[0] 1732428889.336510 trunc from 0x9b1622 to 0x1189fc (0x898c25 bytes) 000069[1] 1732428889.350806 write_OD 0x9e0000 thru 0x9e3fff (0x4000 bytes) 000070[1] 1732428889.363976 write_OD 0x467000 thru 0x472fff (0xc000 bytes) 000072[1] 1732428889.376893 read_OD 0x956000 thru 0x958fff (0x3000 bytes) 000073[0] 1732428889.378491 mapwrite 0x90a000 thru 0x917500 (0xd501 bytes) 000076[0] 1732428889.434974 read_OD 0x6dc000 thru 0x6eafff (0xf000 bytes) 000077[1] 1732428889.446505 mapwrite 0x332000 thru 0x33df91 (0xbf92 bytes) 000078[0] 1732428889.501393 read_OD 0x043000 thru 0x04afff (0x8000 bytes) 000079[1] 1732428889.515533 mapread 0x516000 thru 0x5192fe (0x32ff bytes) 000080[1] 1732428889.521543 mapwrite 0x084000 thru 0x092168 (0xe169 bytes) 000085[0] 1732428889.553976 write_OD 0x966000 thru 0x973fff (0xe000 bytes) 000088[1] 1732428889.569290 trunc from 0x9e4000 to 0x446f17 (0x59d0e8 bytes) 000089[0] 1732428889.577341 write_OD 0x91d000 thru 0x92bfff (0xf000 bytes) 000096[0] 1732428889.595849 read_OD 0x778000 thru 0x783fff (0xc000 bytes) 000097[0] 1732428889.598385 write_OD 0x667000 thru 0x675fff (0xf000 bytes) 000098[0] 1732428889.610655 read_OD 0x134000 thru 0x13efff (0xb000 bytes) 000099[1] 1732428889.613209 write_OD 0x16b000 thru 0x174fff (0xa000 bytes) 000100[0] 1732428889.626079 read_OD 0x6f2000 thru 0x6fffff (0xe000 bytes) 000102[0] 1732428889.629353 write_OD 0x019000 thru 0x027fff (0xf000 bytes) 000103[1] 1732428889.643144 write_OD 0x32a000 thru 0x334fff (0xb000 bytes) 000105[1] 1732428889.655239 mapwrite 0x411000 thru 0x41bd6f (0xad70 bytes) 000106[1] 1732428889.685867 read_OD 0x779000 thru 0x780fff (0x8000 bytes) 000109[1] 1732428889.691040 trunc from 0x92c000 to 0x2c97a6 (0x662859 bytes) 000110[1] 1732428889.697405 mapread 0x013000 thru 0x015b0e (0x2b0f bytes) 000112[1] 1732428889.711150 trunc from 0x2c97a7 to 0x80ea53 (0x5452ad bytes) 000113[0] 1732428889.718139 read_OD 0x5d1000 thru 0x5ddfff (0xd000 bytes) 000116[1] 1732428889.727834 trunc from 0x80ea54 to 0x622a81 (0x1ebfd2 bytes) 000117[0] 1732428889.736283 read_OD 0x3cb000 thru 0x3d1fff (0x7000 bytes) 000118[0] 1732428889.741497 write_OD 0x099000 thru 0x0a1fff (0x9000 bytes) 000120[0] 1732428889.756594 trunc from 0x622a82 to 0x85a8f2 (0x237e71 bytes) 000121[0] 1732428889.765648 mapread 0x670000 thru 0x67cea1 (0xcea2 bytes) 000122[0] 1732428889.805225 mapread 0x004000 thru 0x00d095 (0x9096 bytes) 000125[0] 1732428889.809162 read_OD 0x7ed000 thru 0x7f2fff (0x6000 bytes) 000126[1] 1732428889.820377 mapwrite 0x2b7000 thru 0x2b84c7 (0x14c8 bytes) 000128[1] 1732428889.857663 mapwrite 0x01c000 thru 0x01c1aa (0x1ab bytes) 000132[1] 1732428889.875468 mapread 0x55e000 thru 0x56c2c2 (0xe2c3 bytes) 000133[0] 1732428889.879167 mapwrite 0x0f2000 thru 0x0fcc69 (0xac6a bytes) 000134[1] 1732428889.932157 mapread 0x40f000 thru 0x41de83 (0xee84 bytes) 000135[0] 1732428889.959569 trunc from 0x85a8f3 to 0x199273 (0x6c167f bytes) 000136[0] 1732428889.965453 trunc from 0x199274 to 0x6545ef (0x4bb37c bytes) 000137[0] 1732428889.972507 trunc from 0x6545f0 to 0x79bf80 (0x147991 bytes) 000139[1] 1732428889.982516 trunc from 0x79bf81 to 0x348e65 (0x45311b bytes) 000144[0] 1732428889.996961 read_OD 0x234000 thru 0x23efff (0xb000 bytes) 000150[0] 1732428890.003763 mapwrite 0x545000 thru 0x547d09 (0x2d0a bytes) 000151[0] 1732428890.041136 mapwrite 0x5c7000 thru 0x5c7732 (0x733 bytes) 000154[1] 1732428890.069353 read_OD 0x006000 thru 0x00ffff (0xa000 bytes) 000156[1] 1732428890.079345 mapread 0x2e3000 thru 0x2e64bd (0x34be bytes) 000158[0] 1732428890.106195 mapread 0x46c000 thru 0x4718f2 (0x58f3 bytes) 000161[1] 1732428890.113460 read_OD 0x548000 thru 0x54cfff (0x5000 bytes) 000166[1] 1732428890.116215 mapread 0x239000 thru 0x23c454 (0x3455 bytes) truncating to largest ever: 0x935b5b 000167[0] 1732428890.116650 trunc from 0x5c7733 to 0x935b5a (0x36e428 bytes) 000168[1] 1732428890.128882 write_OD 0x439000 thru 0x439fff (0x1000 bytes) 000169[1] 1732428890.141811 write_OD 0x6da000 thru 0x6dcfff (0x3000 bytes) 000171[0] 1732428890.154933 write_OD 0x98b000 thru 0x999fff (0xf000 bytes) 000173[0] 1732428890.168586 read_OD 0x7ae000 thru 0x7bcfff (0xf000 bytes) 000174[1] 1732428890.171477 write_OD 0x880000 thru 0x882fff (0x3000 bytes) 000175[0] 1732428890.183304 trunc from 0x99a000 to 0x598ffb (0x401004 bytes) 000180[0] 1732428890.199967 mapread 0x4c8000 thru 0x4d5e33 (0xde34 bytes) 000185[0] 1732428890.233939 read_OD 0x350000 thru 0x351fff (0x2000 bytes) 000188[1] 1732428890.242027 mapwrite 0x8c2000 thru 0x8c9b05 (0x7b06 bytes) 000190[1] 1732428890.299864 trunc from 0x8c9b06 to 0x5789c3 (0x351142 bytes) 000191[0] 1732428890.309755 mapread 0x2c7000 thru 0x2c7295 (0x296 bytes) 000193[0] 1732428890.341516 read_OD 0x29f000 thru 0x2acfff (0xe000 bytes) 000194[1] 1732428890.343876 trunc from 0x5789c4 to 0x341e22 (0x236ba1 bytes) 000196[1] 1732428890.358754 write_OD 0x8d8000 thru 0x8dbfff (0x4000 bytes) 000198[1] 1732428890.372111 mapread 0x2b6000 thru 0x2bd0a5 (0x70a6 bytes) 000202[0] 1732428890.383215 read_OD 0x705000 thru 0x705fff (0x1000 bytes) 000203[0] 1732428890.384860 trunc from 0x8dc000 to 0x3d0753 (0x50b8ac bytes) 000204[1] 1732428890.391163 mapread 0x1c8000 thru 0x1d3eb2 (0xbeb3 bytes) 000207[0] 1732428890.409200 write_OD 0x2fe000 thru 0x2fefff (0x1000 bytes) 000209[0] 1732428890.421756 read_OD 0x38b000 thru 0x38ffff (0x5000 bytes) 000210[0] 1732428890.424854 read_OD 0x13e000 thru 0x13efff (0x1000 bytes) 000218[0] 1732428890.426889 mapread 0x121000 thru 0x12b6e3 (0xa6e4 bytes) 000222[0] 1732428890.444942 mapwrite 0x762000 thru 0x7699e0 (0x79e1 bytes) 000224[0] 1732428890.485420 read_OD 0x134000 thru 0x135fff (0x2000 bytes) 000225[1] 1732428890.491721 trunc from 0x7699e1 to 0x4d44ed (0x2954f3 bytes) 000226[1] 1732428890.497689 write_OD 0x3b6000 thru 0x3b9fff (0x4000 bytes) 000227[0] 1732428890.510812 trunc from 0x4d44ee to 0x45ed08 (0x757e5 bytes) 000232[0] 1732428890.527525 trunc from 0x45ed09 to 0x4093ac (0x5595c bytes) 000234[1] 1732428890.532104 mapread 0x236000 thru 0x2412aa (0xb2ab bytes) 000235[0] 1732428890.548938 trunc from 0x4093ad to 0x30c176 (0xfd236 bytes) 000237[1] 1732428890.555293 mapread 0x1bd000 thru 0x1cb1fe (0xe1ff bytes) 000240[0] 1732428890.576726 mapread 0x29e000 thru 0x2ab0a6 (0xd0a7 bytes) 000246[1] 1732428890.594486 mapread 0x16a000 thru 0x176764 (0xc765 bytes) 000248[0] 1732428890.608396 mapwrite 0x8f3000 thru 0x8f4a61 (0x1a62 bytes) 000250[0] 1732428890.656279 mapread 0x8d3000 thru 0x8e0514 (0xd515 bytes) 000255[1] 1732428890.663719 read_OD 0x09a000 thru 0x09cfff (0x3000 bytes) 000258[1] 1732428890.675804 write_OD 0x40f000 thru 0x411fff (0x3000 bytes) 000260[1] 1732428890.688439 mapwrite 0x73b000 thru 0x749c54 (0xec55 bytes) 000267[0] 1732428890.759086 mapwrite 0x56b000 thru 0x57aaa2 (0xfaa3 bytes) 000269[0] 1732428890.799967 trunc from 0x8f4a62 to 0x541f36 (0x3b2b2b bytes) 000274[1] 1732428890.812447 read_OD 0x4b4000 thru 0x4bcfff (0x9000 bytes) 000275[1] 1732428890.818520 write_OD 0x3e8000 thru 0x3edfff (0x6000 bytes) 000276[0] 1732428890.830798 mapread 0x41b000 thru 0x424ab5 (0x9ab6 bytes) 000278[0] 1732428890.841460 trunc from 0x541f37 to 0x7372d2 (0x1f539c bytes) 000280[0] 1732428890.848497 read_OD 0x5a2000 thru 0x5acfff (0xb000 bytes) 000281[0] 1732428890.853505 write_OD 0x669000 thru 0x672fff (0xa000 bytes) 000282[0] 1732428890.866239 mapread 0x5ee000 thru 0x5f3a74 (0x5a75 bytes) 000283[1] 1732428890.891880 read_OD 0x31f000 thru 0x329fff (0xb000 bytes) 000285[0] 1732428890.894300 write_OD 0x3b2000 thru 0x3b2fff (0x1000 bytes) 000286[1] 1732428890.912660 mapread 0x680000 thru 0x68b0c0 (0xb0c1 bytes) 000287[0] 1732428890.942144 read_OD 0x2ed000 thru 0x2effff (0x3000 bytes) 000288[1] 1732428890.943561 write_OD 0x5b1000 thru 0x5b4fff (0x4000 bytes) 000292[0] 1732428890.961887 mapread 0x047000 thru 0x04f143 (0x8144 bytes) 000293[1] 1732428890.989187 write_OD 0x043000 thru 0x04ffff (0xd000 bytes) 000294[1] 1732428891.009381 read_OD 0x495000 thru 0x4a3fff (0xf000 bytes) 000297[1] 1732428891.012797 mapread 0x5c9000 thru 0x5c9359 (0x35a bytes) 000298[1] 1732428891.039993 write_OD 0x9f4000 thru 0x9fcfff (0x9000 bytes) 000301[0] 1732428891.059810 read_OD 0x32c000 thru 0x32cfff (0x1000 bytes) 000302[0] 1732428891.061283 mapread 0x0a1000 thru 0x0acca3 (0xbca4 bytes) 000305[0] 1732428891.088472 read_OD 0x442000 thru 0x44afff (0x9000 bytes) 000306[1] 1732428891.094677 write_OD 0x595000 thru 0x598fff (0x4000 bytes) 000307[1] 1732428891.117140 trunc from 0x9fd000 to 0x527c0c (0x4d53f3 bytes) 000308[0] 1732428891.127163 trunc from 0x527c0d to 0x8f77a2 (0x3cfb96 bytes) 000312[0] 1732428891.143857 read_OD 0x6f5000 thru 0x6fefff (0xa000 bytes) 000318[0] 1732428891.149906 trunc from 0x8f77a3 to 0x44f6ee (0x4a80b4 bytes) 000320[1] 1732428891.163248 mapwrite 0x980000 thru 0x9820e9 (0x20ea bytes) 000321[0] 1732428891.195400 trunc from 0x9820ea to 0x7b9a49 (0x1c86a0 bytes) 000322[0] 1732428891.204438 trunc from 0x7b9a4a to 0x17b80d (0x63e23c bytes) 000327[1] 1732428891.215623 mapwrite 0x91a000 thru 0x91f610 (0x5611 bytes) 000328[1] 1732428891.246798 trunc from 0x91f611 to 0x67f984 (0x29fc8c bytes) 000329[1] 1732428891.251511 mapwrite 0x087000 thru 0x08a38f (0x3390 bytes) 000330[0] 1732428891.267424 mapwrite 0x925000 thru 0x92fe31 (0xae32 bytes) 000335[1] 1732428891.328410 read_OD 0x44d000 thru 0x45afff (0xe000 bytes) truncating to largest ever: 0x9d72c2 000339[0] 1732428891.343678 trunc from 0x92fe32 to 0x9d72c1 (0xa7490 bytes) 000341[0] 1732428891.349859 read_OD 0x47e000 thru 0x489fff (0xc000 bytes) 000343[0] 1732428891.355384 mapwrite 0x0b7000 thru 0x0c0356 (0x9357 bytes) 000344[0] 1732428891.416469 read_OD 0x012000 thru 0x014fff (0x3000 bytes) 000345[1] 1732428891.429975 read_OD 0x5c8000 thru 0x5ccfff (0x5000 bytes) 000346[0] 1732428891.431887 write_OD 0x2bb000 thru 0x2c6fff (0xc000 bytes) 000347[1] 1732428891.445076 trunc from 0x9d72c2 to 0x7ca3e4 (0x20cedd bytes) 000351[0] 1732428891.458819 write_OD 0x490000 thru 0x49bfff (0xc000 bytes) 000352[1] 1732428891.473503 mapwrite 0x700000 thru 0x702536 (0x2537 bytes) 000359[1] 1732428891.521915 write_OD 0x398000 thru 0x39ffff (0x8000 bytes) 000360[1] 1732428891.542022 trunc from 0x7ca3e5 to 0x385faa (0x44443a bytes) 000361[0] 1732428891.551475 trunc from 0x385fab to 0x583681 (0x1fd6d7 bytes) 000362[0] 1732428891.556293 trunc from 0x583682 to 0x3448f5 (0x23ed8c bytes) 000363[1] 1732428891.562096 mapwrite 0x5e7000 thru 0x5f188a (0xa88b bytes) 000364[1] 1732428891.593157 mapread 0x379000 thru 0x383975 (0xa976 bytes) 000366[0] 1732428891.604193 trunc from 0x5f188b to 0x44c2b1 (0x1a55d9 bytes) 000367[0] 1732428891.610260 mapread 0x33f000 thru 0x34c4a6 (0xd4a7 bytes) 000368[0] 1732428891.628614 mapwrite 0x37e000 thru 0x384420 (0x6421 bytes) 000369[1] 1732428891.646965 read_OD 0x3b4000 thru 0x3bcfff (0x9000 bytes) 000370[1] 1732428891.656483 mapread 0x0b4000 thru 0x0c13ad (0xd3ae bytes) 000373[0] 1732428891.676002 mapwrite 0x4ff000 thru 0x4ffd37 (0xd38 bytes) 000374[0] 1732428891.701699 trunc from 0x4ffd38 to 0x72d9ed (0x22dcb6 bytes) 000377[1] 1732428891.709759 write_OD 0x89a000 thru 0x89dfff (0x4000 bytes) 000380[0] 1732428891.722307 mapwrite 0x138000 thru 0x146008 (0xe009 bytes) 000383[1] 1732428891.770082 trunc from 0x89e000 to 0x85b413 (0x42bec bytes) 000389[1] 1732428891.800705 trunc from 0x85b414 to 0x760535 (0xfaede bytes) 000390[0] 1732428891.804990 read_OD 0x480000 thru 0x483fff (0x4000 bytes) 000391[0] 1732428891.811097 mapread 0x599000 thru 0x5a2e9a (0x9e9b bytes) 000393[1] 1732428891.844283 mapread 0x4d7000 thru 0x4dcb56 (0x5b57 bytes) 000397[0] 1732428891.849222 mapwrite 0x273000 thru 0x276bdf (0x3be0 bytes) 000400[1] 1732428891.890666 mapread 0x691000 thru 0x69a0c9 (0x90ca bytes) 000401[0] 1732428891.899485 write_OD 0x5e3000 thru 0x5eafff (0x8000 bytes) 000402[0] 1732428891.911025 read_OD 0x46d000 thru 0x474fff (0x8000 bytes) 000403[0] 1732428891.914279 read_OD 0x2d9000 thru 0x2e3fff (0xb000 bytes) 000404[1] 1732428891.916542 mapwrite 0x2cc000 thru 0x2d6b99 (0xab9a bytes) 000406[1] 1732428891.940302 mapread 0x687000 thru 0x6914b2 (0xa4b3 bytes) 000408[1] 1732428891.951519 mapwrite 0x4cb000 thru 0x4ccb61 (0x1b62 bytes) 000409[0] 1732428891.968019 mapread 0x711000 thru 0x720e7a (0xfe7b bytes) 000410[0] 1732428891.999746 mapwrite 0x1dd000 thru 0x1e1e7e (0x4e7f bytes) 000412[0] 1732428892.029700 trunc from 0x760536 to 0x858879 (0xf8344 bytes) 000413[1] 1732428892.034159 read_OD 0x2f6000 thru 0x2fdfff (0x8000 bytes) 000419[1] 1732428892.042980 read_OD 0x74c000 thru 0x756fff (0xb000 bytes) 000421[0] 1732428892.045584 mapread 0x2d0000 thru 0x2db3ce (0xb3cf bytes) 000422[0] 1732428892.066147 mapwrite 0x5e5000 thru 0x5e78d9 (0x28da bytes) 000423[0] 1732428892.082537 mapwrite 0x37a000 thru 0x3853d3 (0xb3d4 bytes) 000425[1] 1732428892.114715 mapread 0x2a8000 thru 0x2a9f29 (0x1f2a bytes) 000426[1] 1732428892.150093 mapwrite 0x1c3000 thru 0x1ca41d (0x741e bytes) 000429[1] 1732428892.192302 trunc from 0x85887a to 0x564865 (0x2f4014 bytes) 000433[0] 1732428892.208510 write_OD 0x9b3000 thru 0x9bdfff (0xb000 bytes) 000434[1] 1732428892.222118 read_OD 0x488000 thru 0x48cfff (0x5000 bytes) 000436[0] 1732428892.225426 mapread 0x2fe000 thru 0x30c152 (0xe153 bytes) 000439[1] 1732428892.249657 read_OD 0x46f000 thru 0x47afff (0xc000 bytes) 000442[1] 1732428892.252344 write_OD 0x78d000 thru 0x790fff (0x4000 bytes) 000443[1] 1732428892.266255 mapread 0x193000 thru 0x197c5b (0x4c5c bytes) 000444[1] 1732428892.303502 read_OD 0x026000 thru 0x026fff (0x1000 bytes) 000445[1] 1732428892.306280 mapread 0x2b3000 thru 0x2b4c9c (0x1c9d bytes) 000447[0] 1732428892.306676 write_OD 0x8ad000 thru 0x8adfff (0x1000 bytes) 000449[0] 1732428892.326460 mapread 0x10c000 thru 0x114c3b (0x8c3c bytes) 000452[1] 1732428892.337031 write_OD 0x372000 thru 0x374fff (0x3000 bytes) 000453[1] 1732428892.349037 mapwrite 0x47e000 thru 0x481f45 (0x3f46 bytes) 000455[1] 1732428892.408757 trunc from 0x9be000 to 0x7584a2 (0x265b5d bytes) 000456[0] 1732428892.415722 mapread 0x35d000 thru 0x366ca8 (0x9ca9 bytes) 000458[0] 1732428892.437642 write_OD 0x9e7000 thru 0x9e9fff (0x3000 bytes) 000459[0] 1732428892.451610 mapread 0x17f000 thru 0x1839dc (0x49dd bytes) 000460[0] 1732428892.460326 write_OD 0x60d000 thru 0x60dfff (0x1000 bytes) 000461[1] 1732428892.472415 mapwrite 0x5a3000 thru 0x5ae978 (0xb979 bytes) 000462[1] 1732428892.502135 read_OD 0x1a8000 thru 0x1a9fff (0x2000 bytes) 000463[0] 1732428892.505900 trunc from 0x9ea000 to 0x897fea (0x152015 bytes) 000465[0] 1732428892.516210 mapread 0x232000 thru 0x23f81d (0xd81e bytes) 000466[0] 1732428892.533833 trunc from 0x897feb to 0x1bd112 (0x6daed8 bytes) 000468[1] 1732428892.538875 mapread 0x001000 thru 0x00a6d7 (0x96d8 bytes) 000471[1] 1732428892.556822 write_OD 0x4ba000 thru 0x4c3fff (0xa000 bytes) 000472[0] 1732428892.572852 read_OD 0x3a7000 thru 0x3abfff (0x5000 bytes) 000474[0] 1732428892.575931 trunc from 0x4c4000 to 0x9d26e1 (0x50e6e2 bytes) 000475[1] 1732428892.586309 write_OD 0x584000 thru 0x586fff (0x3000 bytes) 000476[1] 1732428892.598748 read_OD 0x714000 thru 0x71bfff (0x8000 bytes) 000478[0] 1732428892.602004 mapread 0x23e000 thru 0x23e343 (0x344 bytes) 000479[0] 1732428892.640006 write_OD 0x28b000 thru 0x297fff (0xd000 bytes) 000480[1] 1732428892.660550 mapwrite 0x3b1000 thru 0x3b4d87 (0x3d88 bytes) 000482[0] 1732428892.722281 write_OD 0x16e000 thru 0x170fff (0x3000 bytes) 000483[1] 1732428892.742342 read_OD 0x042000 thru 0x044fff (0x3000 bytes) 000484[1] 1732428892.744887 mapwrite 0x0fc000 thru 0x107e6d (0xbe6e bytes) 000485[0] 1732428892.801883 write_OD 0x37e000 thru 0x382fff (0x5000 bytes) 000490[1] 1732428892.824035 write_OD 0x2e2000 thru 0x2f0fff (0xf000 bytes) 000491[1] 1732428892.837785 trunc from 0x9d26e2 to 0x5180ee (0x4ba5f3 bytes) 000493[1] 1732428892.847395 read_OD 0x213000 thru 0x21afff (0x8000 bytes) 000494[0] 1732428892.851197 write_OD 0x117000 thru 0x119fff (0x3000 bytes) 000495[1] 1732428892.861776 mapread 0x419000 thru 0x42762e (0xe62f bytes) 000498[1] 1732428892.883766 write_OD 0x26f000 thru 0x27cfff (0xe000 bytes) 000499[0] 1732428892.903904 mapwrite 0x1cc000 thru 0x1d2b61 (0x6b62 bytes) 000500[0] 1732428892.931006 mapread 0x263000 thru 0x26318a (0x18b bytes) All operations completed A-OK! PASS 16a (10s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 01:14:54 (1732428894) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.117158 s, 71.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.109052 s, 76.9 MB/s Chance of close/open is 1 in 50 Seed set to 8894 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x8c0434 000003[0] 1732428894.863266 trunc from 00000000 to 0x8c0433 (0x8c0434 bytes) 000004[1] 1732428894.875645 mapwrite 0x9679c1 thru 0x96871c (0xd5c bytes) 000005[1] 1732428894.910872 mapread 0x1927d4 thru 0x19323e (0xa6b bytes) 000007[1] 1732428894.914528 mapread 0x3a223c thru 0x3a2400 (0x1c5 bytes) 000008[1] 1732428894.916009 read 0x7d6618 thru 0x7d68d7 (0x2c0 bytes) 000009[1] 1732428894.917255 mapwrite 0x6279ae thru 0x6284e9 (0xb3c bytes) 000010[1] 1732428894.933852 mapwrite 0x8794da thru 0x879b1a (0x641 bytes) 000013[1] 1732428894.953203 mapread 0x6d3f95 thru 0x6d5b6f (0x1bdb bytes) 000015[0] 1732428894.963993 mapread 0x78bbb2 thru 0x78c38b (0x7da bytes) 000016[1] 1732428894.970998 read 0x4583bd thru 0x458628 (0x26c bytes) 000021[0] 1732428895.015654 write 0x16b104 thru 0x16cb25 (0x1a22 bytes) truncating to largest ever: 0x99a247 000023[1] 1732428895.027503 trunc from 0x96871d to 0x99a246 (0x31b2a bytes) 000025[1] 1732428895.047485 mapread 0x0787ee thru 0x078c58 (0x46b bytes) 000026[1] 1732428895.063047 write 0x578b26 thru 0x57a691 (0x1b6c bytes) 000027[1] 1732428895.066765 trunc from 0x99a247 to 0x946491 (0x53db5 bytes) 000030[0] 1732428895.076785 mapwrite 0x9b7d0f thru 0x9b8072 (0x364 bytes) 000031[1] 1732428895.105206 write 0x87efe4 thru 0x87fbcf (0xbec bytes) 000032[1] 1732428895.114470 trunc from 0x9b8073 to 0x8e2d84 (0xd52ee bytes) 000033[1] 1732428895.124041 mapread 0x3147ef thru 0x3164a2 (0x1cb4 bytes) 000034[1] 1732428895.128857 write 0x0bf61d thru 0x0bfb52 (0x536 bytes) 000037[1] 1732428895.134127 write 0x93d578 thru 0x93dbfc (0x685 bytes) 000039[0] 1732428895.141537 trunc from 0x93dbfd to 0x8f244f (0x4b7ad bytes) 000040[0] 1732428895.154555 mapwrite 0x1a80a8 thru 0x1a9a6e (0x19c7 bytes) 000045[1] 1732428895.178288 write 0x0cbfb8 thru 0x0cc919 (0x962 bytes) 000046[0] 1732428895.186234 mapwrite 0x0798eb thru 0x07a341 (0xa57 bytes) 000047[0] 1732428895.204177 mapwrite 0x573065 thru 0x574c6b (0x1c07 bytes) 000050[1] 1732428895.227085 write 0x45e589 thru 0x45fd52 (0x17ca bytes) 000054[1] 1732428895.244648 trunc from 0x8f2450 to 0x1569c0 (0x79ba8f bytes) 000057[0] 1732428895.256550 mapwrite 0x6ec86e thru 0x6ed507 (0xc9a bytes) 000058[0] 1732428895.278333 trunc from 0x6ed508 to 0x0e0038 (0x60d4cf bytes) 000060[1] 1732428895.285841 mapread 0x042d43 thru 0x0449c0 (0x1c7e bytes) 000062[1] 1732428895.293579 mapread 0x05e8e6 thru 0x060154 (0x186f bytes) 000063[0] 1732428895.298518 trunc from 0x0e0039 to 0x4d89e1 (0x3f89a9 bytes) 000064[1] 1732428895.308289 mapread 0x482f3f thru 0x4835a6 (0x668 bytes) 000065[1] 1732428895.316341 read 0x3ca871 thru 0x3cb828 (0xfb8 bytes) 000066[1] 1732428895.339735 read 0x359a5f thru 0x35a623 (0xbc5 bytes) 000067[0] 1732428895.341936 mapread 0x10bce1 thru 0x10c91b (0xc3b bytes) 000068[1] 1732428895.346497 trunc from 0x4d89e2 to 0x142206 (0x3967db bytes) 000069[1] 1732428895.364118 trunc from 0x142207 to 0x1adf89 (0x6bd83 bytes) 000072[0] 1732428895.375783 read 0x0a1294 thru 0x0a2026 (0xd93 bytes) 000075[1] 1732428895.384277 mapread 0x129222 thru 0x129663 (0x442 bytes) 000078[1] 1732428895.402630 write 0x11d174 thru 0x11daa1 (0x92e bytes) 000081[0] 1732428895.417519 read 0x0e4003 thru 0x0e5d22 (0x1d20 bytes) 000084[0] 1732428895.439575 write 0x29de26 thru 0x29ef63 (0x113e bytes) 000085[1] 1732428895.447103 write 0x454911 thru 0x454ec0 (0x5b0 bytes) 000087[1] 1732428895.455071 read 0x2143ee thru 0x215591 (0x11a4 bytes) 000088[0] 1732428895.477647 read 0x01ff02 thru 0x021c7d (0x1d7c bytes) 000089[0] 1732428895.495457 read 0x29ffb8 thru 0x2a0abe (0xb07 bytes) 000090[0] 1732428895.504479 read 0x1558f8 thru 0x1569b0 (0x10b9 bytes) 000093[0] 1732428895.505049 read 0x2fa92b thru 0x2fb4eb (0xbc1 bytes) 000100[1] 1732428895.520724 trunc from 0x454ec1 to 0x68726a (0x2323aa bytes) 000101[0] 1732428895.529775 read 0x680acc thru 0x6811f2 (0x727 bytes) 000105[1] 1732428895.553786 mapwrite 0x894659 thru 0x89485d (0x205 bytes) 000106[0] 1732428895.586840 write 0x9c3119 thru 0x9c36dd (0x5c5 bytes) 000107[0] 1732428895.595247 trunc from 0x9c36de to 0x6e0168 (0x2e3575 bytes) 000109[0] 1732428895.602733 write 0x354e20 thru 0x355c32 (0xe13 bytes) 000113[1] 1732428895.614705 read 0x4d5bc3 thru 0x4d6436 (0x874 bytes) 000115[0] 1732428895.626184 mapread 0x2708e6 thru 0x2709d7 (0xf2 bytes) 000116[0] 1732428895.629641 mapread 0x6572ae thru 0x657e54 (0xba7 bytes) 000117[0] 1732428895.631092 trunc from 0x6e0169 to 0x894340 (0x1b41d8 bytes) 000118[0] 1732428895.641092 write 0x730754 thru 0x732235 (0x1ae2 bytes) 000121[1] 1732428895.660839 write 0x299d4c thru 0x29aa78 (0xd2d bytes) 000125[1] 1732428895.697968 mapwrite 0x05a2db thru 0x05ab2f (0x855 bytes) 000126[0] 1732428895.714326 mapwrite 0x27b4ce thru 0x27d2ef (0x1e22 bytes) 000127[1] 1732428895.745612 trunc from 0x894341 to 0x2d4d4a (0x5bf5f6 bytes) 000128[1] 1732428895.757194 write 0x8bcc4d thru 0x8bda83 (0xe37 bytes) 000130[1] 1732428895.763484 mapread 0x1f6487 thru 0x1f730d (0xe87 bytes) 000132[0] 1732428895.777706 read 0x696902 thru 0x698576 (0x1c75 bytes) 000136[0] 1732428895.798789 mapwrite 0x8b293e thru 0x8b2d70 (0x433 bytes) 000137[0] 1732428895.817857 mapread 0x3aa627 thru 0x3abce6 (0x16c0 bytes) 000138[1] 1732428895.820726 mapwrite 0x67aab2 thru 0x67c3e1 (0x1930 bytes) 000139[0] 1732428895.852357 write 0x42247f thru 0x424212 (0x1d94 bytes) 000140[1] 1732428895.860396 mapwrite 0x24b026 thru 0x24b51c (0x4f7 bytes) 000142[1] 1732428895.882702 read 0x64ee3d thru 0x650690 (0x1854 bytes) 000144[1] 1732428895.909529 mapread 0x3f898f thru 0x3f90de (0x750 bytes) 000146[0] 1732428895.916532 mapread 0x58684d thru 0x5871e3 (0x997 bytes) 000147[0] 1732428895.920548 mapread 0x44ce84 thru 0x44e6eb (0x1868 bytes) 000148[0] 1732428895.926381 mapread 0x7ddbdd thru 0x7ddf45 (0x369 bytes) 000149[0] 1732428895.934079 read 0x5c7f46 thru 0x5c835e (0x419 bytes) 000150[1] 1732428895.939722 read 0x0bb354 thru 0x0bcde7 (0x1a94 bytes) 000151[0] 1732428895.940127 mapwrite 0x34f15a thru 0x350889 (0x1730 bytes) 000152[1] 1732428895.986843 read 0x50ecd7 thru 0x510745 (0x1a6f bytes) 000156[1] 1732428895.994226 mapread 0x7c22b9 thru 0x7c3f83 (0x1ccb bytes) 000158[1] 1732428896.005100 read 0x7cf2b1 thru 0x7cff1e (0xc6e bytes) 000159[1] 1732428896.012759 write 0x7d2822 thru 0x7d330d (0xaec bytes) 000160[0] 1732428896.020156 mapwrite 0x0eb308 thru 0x0eb3e0 (0xd9 bytes) 000161[0] 1732428896.036011 read 0x57b55f thru 0x57c425 (0xec7 bytes) 000162[1] 1732428896.058173 read 0x169347 thru 0x1694dd (0x197 bytes) 000163[0] 1732428896.084725 read 0x7d1e70 thru 0x7d2130 (0x2c1 bytes) 000164[1] 1732428896.085010 read 0x01e55d thru 0x01fb1f (0x15c3 bytes) 000166[0] 1732428896.086202 mapwrite 0x5d1897 thru 0x5d265e (0xdc8 bytes) 000167[1] 1732428896.138628 write 0x008ea7 thru 0x009dc1 (0xf1b bytes) 000168[1] 1732428896.167561 mapwrite 0x93ee01 thru 0x93fa88 (0xc88 bytes) 000172[1] 1732428896.213935 mapread 0x089f74 thru 0x08b38f (0x141c bytes) 000176[0] 1732428896.220634 mapread 0x6d34c7 thru 0x6d3554 (0x8e bytes) 000177[1] 1732428896.235221 mapread 0x44af98 thru 0x44cdac (0x1e15 bytes) 000178[0] 1732428896.239522 mapwrite 0x818b6b thru 0x81937d (0x813 bytes) 000181[1] 1732428896.272800 mapwrite 0x0c8a9b thru 0x0c9b93 (0x10f9 bytes) 000183[0] 1732428896.297736 mapwrite 0x0388a4 thru 0x039731 (0xe8e bytes) 000184[0] 1732428896.316781 write 0x8bf624 thru 0x8bf9f6 (0x3d3 bytes) 000186[0] 1732428896.321491 write 0x967e1d thru 0x9695a0 (0x1784 bytes) 000189[0] 1732428896.322391 read 0x7b28ac thru 0x7b443c (0x1b91 bytes) 000191[0] 1732428896.354980 write 0x61137a thru 0x611d04 (0x98b bytes) 000192[0] 1732428896.365242 write 0x8fc59e thru 0x8fced0 (0x933 bytes) 000194[1] 1732428896.365962 write 0x36bfeb thru 0x36bffd (0x13 bytes) 000199[0] 1732428896.393819 mapwrite 0x2d6ef5 thru 0x2d83a3 (0x14af bytes) 000205[1] 1732428896.426699 mapread 0x496b76 thru 0x497dc8 (0x1253 bytes) 000208[1] 1732428896.441433 trunc from 0x9695a1 to 0x7fe057 (0x16b549 bytes) 000210[1] 1732428896.455442 write 0x90e740 thru 0x90eace (0x38f bytes) 000212[1] 1732428896.460414 trunc from 0x90eacf to 0x95d4eb (0x4ea1d bytes) 000213[0] 1732428896.468345 trunc from 0x95d4ec to 0x1cd36d (0x79017e bytes) 000214[0] 1732428896.480796 trunc from 0x1cd36e to 0x8cfa4b (0x7026de bytes) 000216[0] 1732428896.487233 mapwrite 0x6e710f thru 0x6e7e1c (0xd0e bytes) 000217[0] 1732428896.504617 read 0x13232d thru 0x134216 (0x1eea bytes) 000219[0] 1732428896.538115 read 0x302ba5 thru 0x302d04 (0x160 bytes) 000220[0] 1732428896.544190 mapwrite 0x0ea2fa thru 0x0eb01d (0xd24 bytes) 000221[1] 1732428896.556304 trunc from 0x8cfa4c to 0x476f7e (0x458acd bytes) 000222[1] 1732428896.576638 write 0x765936 thru 0x766dcb (0x1496 bytes) 000223[0] 1732428896.578908 mapread 0x43f648 thru 0x44066f (0x1028 bytes) 000225[1] 1732428896.590486 write 0x004ea5 thru 0x006be7 (0x1d43 bytes) 000228[0] 1732428896.596187 mapread 0x198204 thru 0x1991ef (0xfec bytes) 000234[1] 1732428896.623585 trunc from 0x766dcc to 0x6931d4 (0xd3bf7 bytes) 000236[1] 1732428896.638619 read 0x206abc thru 0x208083 (0x15c8 bytes) 000237[1] 1732428896.660540 mapread 0x05641d thru 0x05659e (0x182 bytes) 000238[0] 1732428896.660990 mapread 0x5bf0a6 thru 0x5c0c5e (0x1bb9 bytes) 000240[1] 1732428896.693841 write 0x152765 thru 0x153751 (0xfed bytes) 000242[0] 1732428896.697525 read 0x5b7081 thru 0x5b883f (0x17bf bytes) 000243[1] 1732428896.697844 trunc from 0x6931d5 to 0x939a6e (0x2a689a bytes) 000245[1] 1732428896.712997 trunc from 0x939a6f to 0x87b4b5 (0xbe5b9 bytes) 000247[0] 1732428896.718268 mapwrite 0x3cd205 thru 0x3cd75d (0x559 bytes) 000248[0] 1732428896.761489 mapread 0x873b59 thru 0x874899 (0xd41 bytes) 000249[1] 1732428896.769809 read 0x4a83fc thru 0x4a852f (0x134 bytes) 000250[1] 1732428896.774309 mapread 0x68498d thru 0x6849b4 (0x28 bytes) 000252[0] 1732428896.783935 write 0x7307f9 thru 0x731bd8 (0x13e0 bytes) 000255[0] 1732428896.809527 trunc from 0x87b4b6 to 0x6b2a4b (0x1c8a6a bytes) 000256[0] 1732428896.818614 trunc from 0x6b2a4c to 0x79d636 (0xeabeb bytes) 000257[1] 1732428896.823139 write 0x651d38 thru 0x653c45 (0x1f0e bytes) 000259[1] 1732428896.837562 mapread 0x4ba40e thru 0x4bac6c (0x85f bytes) 000261[1] 1732428896.846257 mapwrite 0x5eeb54 thru 0x5f029d (0x174a bytes) 000263[0] 1732428896.868196 read 0x75ef8e thru 0x760a82 (0x1af5 bytes) 000264[0] 1732428896.893847 read 0x3c3f09 thru 0x3c420f (0x307 bytes) 000265[0] 1732428896.894224 read 0x6cc170 thru 0x6ccb95 (0xa26 bytes) 000266[1] 1732428896.894537 trunc from 0x79d637 to 0x64c41c (0x15121a bytes) 000267[1] 1732428896.912166 read 0x24684e thru 0x247d3f (0x14f2 bytes) 000270[0] 1732428896.946228 mapwrite 0x45fa33 thru 0x45fc58 (0x226 bytes) 000271[0] 1732428896.967202 read 0x2f113c thru 0x2f2b62 (0x1a27 bytes) 000273[1] 1732428896.998507 read 0x499c7d thru 0x49aa0a (0xd8e bytes) 000274[1] 1732428897.028583 mapread 0x428e3c thru 0x42928c (0x451 bytes) 000280[1] 1732428897.029958 mapread 0x5b0e81 thru 0x5b13d8 (0x558 bytes) 000281[0] 1732428897.030309 read 0x22d932 thru 0x22f8c7 (0x1f96 bytes) 000283[0] 1732428897.030670 read 0x56c12a thru 0x56d1f0 (0x10c7 bytes) 000286[0] 1732428897.045166 read 0x1e5554 thru 0x1e7418 (0x1ec5 bytes) 000288[0] 1732428897.045579 mapwrite 0x1d101e thru 0x1d1207 (0x1ea bytes) 000291[1] 1732428897.096729 write 0x02b57e thru 0x02d07b (0x1afe bytes) 000294[1] 1732428897.111022 mapread 0x1af1ef thru 0x1b07ef (0x1601 bytes) 000296[0] 1732428897.119441 trunc from 0x64c41d to 0x98017c (0x333d60 bytes) 000297[0] 1732428897.130583 read 0x63b072 thru 0x63c55a (0x14e9 bytes) 000298[1] 1732428897.168976 mapwrite 0x2afdf4 thru 0x2b19b5 (0x1bc2 bytes) 000299[1] 1732428897.196999 trunc from 0x98017d to 0x8cd00a (0xb3172 bytes) 000302[1] 1732428897.214875 trunc from 0x8cd00b to 0x73f764 (0x18d8a6 bytes) 000303[1] 1732428897.222522 mapread 0x59b8eb thru 0x59d06e (0x1784 bytes) 000308[1] 1732428897.243098 trunc from 0x73f765 to 0x739c9e (0x5ac6 bytes) 000309[0] 1732428897.247259 read 0x53eff9 thru 0x540b0f (0x1b17 bytes) 000310[1] 1732428897.257784 write 0x80752b thru 0x807a65 (0x53b bytes) 000313[0] 1732428897.259507 read 0x3b0b6c thru 0x3b1cf2 (0x1187 bytes) 000314[1] 1732428897.284396 write 0x2635d0 thru 0x2654ac (0x1edd bytes) 000317[1] 1732428897.300839 write 0x498fd0 thru 0x49a0e2 (0x1113 bytes) 000318[0] 1732428897.308837 mapread 0x804fa1 thru 0x805337 (0x397 bytes) 000321[1] 1732428897.320902 mapread 0x4367f2 thru 0x437ad4 (0x12e3 bytes) 000322[1] 1732428897.327490 mapwrite 0x5ee660 thru 0x5efa09 (0x13aa bytes) 000323[0] 1732428897.341365 mapread 0x0a5d5f thru 0x0a7b41 (0x1de3 bytes) 000327[1] 1732428897.346841 write 0x82381b thru 0x824c9f (0x1485 bytes) 000328[0] 1732428897.349158 mapwrite 0x380977 thru 0x381d12 (0x139c bytes) 000331[1] 1732428897.377489 read 0x3e4a4a thru 0x3e4d79 (0x330 bytes) 000333[0] 1732428897.401178 write 0x4f1e6c thru 0x4f3ccb (0x1e60 bytes) 000334[1] 1732428897.408830 read 0x2ec65a thru 0x2ecc34 (0x5db bytes) 000335[1] 1732428897.409226 trunc from 0x824ca0 to 0x33c2d0 (0x4e89cf bytes) 000336[0] 1732428897.422566 mapwrite 0x38a8dd thru 0x38c16b (0x188f bytes) 000338[1] 1732428897.451004 trunc from 0x38c16c to 0x390357 (0x41ec bytes) 000342[0] 1732428897.469518 write 0x1da46d thru 0x1db4f1 (0x1085 bytes) 000344[0] 1732428897.480482 mapread 0x14439f thru 0x145fa9 (0x1c0b bytes) 000345[1] 1732428897.488018 trunc from 0x390358 to 0x8c49a1 (0x53464a bytes) 000347[0] 1732428897.503241 write 0x160725 thru 0x1619e9 (0x12c5 bytes) 000348[1] 1732428897.510689 trunc from 0x8c49a2 to 0x36011d (0x564884 bytes) 000349[1] 1732428897.521758 trunc from 0x36011e to 0x2729b8 (0xed765 bytes) 000350[0] 1732428897.526139 trunc from 0x2729b9 to 0x4d4f5c (0x2625a4 bytes) 000352[1] 1732428897.538422 read 0x3fe02e thru 0x3fe052 (0x25 bytes) 000354[0] 1732428897.558533 mapread 0x17d415 thru 0x17e918 (0x1504 bytes) 000355[1] 1732428897.564483 write 0x0fa1c5 thru 0x0fbcd7 (0x1b13 bytes) 000356[0] 1732428897.575519 mapread 0x02a2a5 thru 0x02bca5 (0x1a01 bytes) 000357[1] 1732428897.582287 mapread 0x3908c7 thru 0x391271 (0x9ab bytes) 000358[1] 1732428897.586357 mapread 0x0833fb thru 0x0846ef (0x12f5 bytes) 000359[1] 1732428897.593746 write 0x64ef3f thru 0x64fc34 (0xcf6 bytes) 000360[1] 1732428897.596258 trunc from 0x64fc35 to 0x617b0b (0x38129 bytes) 000361[0] 1732428897.601796 mapread 0x15c46d thru 0x15d2b5 (0xe49 bytes) 000365[0] 1732428897.616707 trunc from 0x617b0c to 0x133c34 (0x4e3ed7 bytes) 000366[0] 1732428897.628105 mapread 0x12a475 thru 0x12b0a2 (0xc2e bytes) 000367[1] 1732428897.631786 mapwrite 0x98a540 thru 0x98adb5 (0x876 bytes) 000369[0] 1732428897.669260 mapread 0x30cc84 thru 0x30e6d9 (0x1a56 bytes) 000371[0] 1732428897.682583 trunc from 0x98adb6 to 0x2f9500 (0x6918b5 bytes) 000372[1] 1732428897.695921 mapread 0x20e8a6 thru 0x20f7bc (0xf17 bytes) 000373[1] 1732428897.703950 mapread 0x0c17b4 thru 0x0c2cd8 (0x1525 bytes) 000375[0] 1732428897.711912 mapread 0x04e8b3 thru 0x0501f1 (0x193f bytes) 000378[0] 1732428897.725906 trunc from 0x2f9501 to 0x1f32e8 (0x106218 bytes) 000379[1] 1732428897.736739 trunc from 0x1f32e9 to 0x7477a6 (0x5544be bytes) 000380[0] 1732428897.745217 trunc from 0x7477a7 to 0x68d6d0 (0xba0d6 bytes) 000382[0] 1732428897.753552 trunc from 0x68d6d1 to 0x555711 (0x137fbf bytes) 000383[0] 1732428897.757199 mapwrite 0x1c4251 thru 0x1c5fa4 (0x1d54 bytes) 000386[1] 1732428897.771643 mapwrite 0x4a7863 thru 0x4a798b (0x129 bytes) 000389[1] 1732428897.797733 write 0x52eac2 thru 0x52f429 (0x968 bytes) 000392[0] 1732428897.810004 trunc from 0x555712 to 0x5128ec (0x42e25 bytes) 000393[1] 1732428897.820722 read 0x20eed9 thru 0x20f6d4 (0x7fc bytes) 000397[0] 1732428897.855433 trunc from 0x5128ed to 0x7f1735 (0x2dee49 bytes) 000400[0] 1732428897.870323 write 0x940c9b thru 0x9420b5 (0x141b bytes) 000403[1] 1732428897.877967 mapread 0x2e4051 thru 0x2e550d (0x14bd bytes) 000404[0] 1732428897.886635 mapread 0x51abad thru 0x51b668 (0xabc bytes) 000405[0] 1732428897.895749 mapread 0x94185d thru 0x9420b5 (0x859 bytes) 000407[1] 1732428897.902089 mapread 0x45796a thru 0x4590e4 (0x177b bytes) 000408[0] 1732428897.926224 write 0x628baa thru 0x62a0fe (0x1555 bytes) 000409[1] 1732428897.937849 trunc from 0x9420b6 to 0x0f96f7 (0x8489be bytes) 000410[1] 1732428897.947404 read 0x0849f2 thru 0x085e78 (0x1487 bytes) 000411[0] 1732428897.956637 trunc from 0x0f96f8 to 0x0bfd9d (0x3995a bytes) 000413[0] 1732428897.971377 mapread 0x0b39e7 thru 0x0b45fe (0xc18 bytes) truncating to largest ever: 0x9ebf85 000414[1] 1732428897.977700 trunc from 0x0bfd9e to 0x9ebf84 (0x92c1e7 bytes) 000415[1] 1732428897.986671 mapread 0x630750 thru 0x63093c (0x1ed bytes) 000416[1] 1732428897.996099 mapwrite 0x58af45 thru 0x58c6c3 (0x177f bytes) 000419[0] 1732428898.037755 mapwrite 0x007124 thru 0x00872b (0x1608 bytes) 000420[1] 1732428898.061765 mapread 0x0ffd64 thru 0x10187c (0x1b19 bytes) 000423[1] 1732428898.082495 mapread 0x00680e thru 0x006c5b (0x44e bytes) 000428[0] 1732428898.090753 read 0x6af4bd thru 0x6af78c (0x2d0 bytes) 000429[1] 1732428898.122957 mapread 0x90cee4 thru 0x90ee74 (0x1f91 bytes) 000431[0] 1732428898.123526 mapread 0x74e425 thru 0x74f62a (0x1206 bytes) 000432[0] 1732428898.123815 read 0x09480e thru 0x0961d9 (0x19cc bytes) 000434[0] 1732428898.124185 read 0x9a9093 thru 0x9ab08f (0x1ffd bytes) 000436[1] 1732428898.135881 mapwrite 0x4cc4c8 thru 0x4cc856 (0x38f bytes) 000441[0] 1732428898.165811 write 0x726598 thru 0x726692 (0xfb bytes) 000443[1] 1732428898.176352 mapread 0x87b25a thru 0x87c9d6 (0x177d bytes) 000447[1] 1732428898.182340 write 0x93a94d thru 0x93aee8 (0x59c bytes) 000449[1] 1732428898.199271 trunc from 0x9ebf85 to 0x1de67a (0x80d90a bytes) 000454[0] 1732428898.212541 mapwrite 0x448a8c thru 0x449c79 (0x11ee bytes) 000455[1] 1732428898.237996 write 0x400213 thru 0x401544 (0x1332 bytes) 000459[0] 1732428898.256505 mapwrite 0x8dd915 thru 0x8ddbb7 (0x2a3 bytes) 000460[0] 1732428898.278991 mapread 0x64bce6 thru 0x64c636 (0x951 bytes) 000462[0] 1732428898.282399 mapwrite 0x41ca46 thru 0x41ccb9 (0x274 bytes) 000463[1] 1732428898.299305 write 0x7b363c thru 0x7b5019 (0x19de bytes) 000464[0] 1732428898.306927 trunc from 0x8ddbb8 to 0x6a4218 (0x23999f bytes) 000465[1] 1732428898.318677 mapwrite 0x687dea thru 0x689db8 (0x1fcf bytes) 000466[1] 1732428898.340023 trunc from 0x6a4219 to 0x638ba4 (0x6b674 bytes) 000467[0] 1732428898.346039 mapread 0x4d5a04 thru 0x4d7290 (0x188d bytes) 000471[0] 1732428898.363135 mapread 0x19cd75 thru 0x19dae3 (0xd6f bytes) 000474[1] 1732428898.370295 write 0x3a972b thru 0x3ab3a0 (0x1c76 bytes) 000475[0] 1732428898.377616 write 0x34f392 thru 0x34f7c7 (0x436 bytes) 000476[0] 1732428898.382604 trunc from 0x638ba5 to 0x392a58 (0x2a614c bytes) 000477[0] 1732428898.388950 trunc from 0x392a59 to 0x1a1c4a (0x1f0e0e bytes) 000478[1] 1732428898.396045 mapread 0x071bc1 thru 0x0721ba (0x5fa bytes) 000480[0] 1732428898.408867 read 0x162d78 thru 0x163211 (0x49a bytes) 000482[1] 1732428898.421891 read 0x185aef thru 0x186c3f (0x1151 bytes) 000487[0] 1732428898.437580 write 0x8ce9e3 thru 0x8cfdf3 (0x1411 bytes) 000492[0] 1732428898.453638 read 0x4bc528 thru 0x4bcb79 (0x652 bytes) 000496[1] 1732428898.486100 mapread 0x4a1576 thru 0x4a188c (0x317 bytes) 000498[1] 1732428898.502748 trunc from 0x8cfdf4 to 0x0695b3 (0x866840 bytes) All operations completed A-OK! PASS 16b (5s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 01:14:59 (1732428899) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 01:15:00 (1732428900) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.113456 s, 73.9 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.110238 s, 76.1 MB/s ldlm.namespaces.MGC192.168.203.156@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012a4fe800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0d7800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012a4fe800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012d0d7800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012a4fe800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012d0d7800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.3064 s, 45.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.62906 s, 167 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.561935 s, 187 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.23005 s, 85.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.69413 s, 61.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.879311 s, 119 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 01:15:10 (1732428910) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0525566 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030323 s, 34.6 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 01:15:12 (1732428912) TIMEOUT PASS 16f (21s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 01:15:33 (1732428933) TIMEOUT PASS 16g (22s) == sanityn test 16h: mmap read after truncate file ======= 01:15:55 (1732428955) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) == sanityn test 16i: read after truncate file ============ 01:15:57 (1732428957) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 16j: race dio with buffered i/o ========== 01:15:58 (1732428958) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.354525 s, 73.9 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.277932 s, 94.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.475163 s, 55.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.167274 s, 157 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.428355 s, 61.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.165616 s, 158 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.461898 s, 56.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.322807 s, 81.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.335809 s, 78.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.540621 s, 48.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.559029 s, 46.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.358752 s, 73.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.370696 s, 70.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.513453 s, 51.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.52563 s, 49.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.252757 s, 104 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.352567 s, 74.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.572445 s, 45.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.61905 s, 42.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (7s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 01:16:05 (1732428965) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported Operation not supported skipping zero size read skipping zero size read skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Sun Nov 24 01:16:26 EST 2024 All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Sun Nov 24 01:16:27 EST 2024 + fsx 3 done: rc=0 Sun Nov 24 01:16:27 EST 2024 + fsx 4 done: rc=0 Sun Nov 24 01:16:27 EST 2024 + fsx 5 done: rc=0 Sun Nov 24 01:16:27 EST 2024 PASS 16k (23s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 01:16:28 (1732428988) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 01:16:32 (1732428992) mmap test1: basic mmap operation (PASS, 0.020001s) mmap test2: MAP_PRIVATE not write back (PASS, 0.014052s) mmap test3: concurrent mmap ops on two nodes (PASS, 15.603s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.0748s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.028078s) mmap test6: check mmap write/read content on two nodes (PASS, 0.034859s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.30474s) mmap test11: multi-thread shared mmap (PASS, 0.333s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0128s) PASS 18 (24s) == sanityn test 19: test concurrent uncached read races ========================================================================= 01:16:56 (1732429016) oleg356-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 01:16:57 (1732429017) PASS 20 (1s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 01:16:58 (1732429018) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 01:17:00 (1732429020) now is 1732429020 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 new atime is 1732429081 PASS 23 (62s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 01:18:02 (1732429082) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 386 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 386 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 31.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 39.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234850 386 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 386 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre[MDT:0] filesystem_summary: 502242 386 501856 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234850 386 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 01:18:04 (1732429084) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 01:18:05 (1732429085) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 01:18:07 (1732429087) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (0s) == sanityn test 26a: allow mtime to get older ============ 01:18:07 (1732429087) utime: good mknod times 1732429087 <= 1732429088 <= 1732429088 for /mnt/lustre/f26a utime: good mknod times 1732429087 <= 1732429088 <= 1732429088 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 01:18:10 (1732429090) PASS 26b (3s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 01:18:13 (1732429093) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2024-11-24 01:18:14.000000000 -0500 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2024-11-24 01:18:14.000000000 -0500 Birth: - PASS 26c (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 01:18:16 (1732429096) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 01:18:20 (1732429100) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 01:18:22 (1732429102) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg356-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.156@tcp:/lustre 7542784 39936 7496704 1% /mnt/lustre2 PASS 31b (10s) == sanityn test 31r: open-rename(replace) race =========== 01:18:32 (1732429112) -rw-r--r-- 1 root root 0 Nov 24 01:18 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 01:18:37 (1732429117) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13107 Terminated tail -f $tfile PASS 31s (1s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 01:18:38 (1732429118) PASS 31t (3s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 01:18:41 (1732429121) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 01:18:42 (1732429122) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 01:18:42 (1732429122) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 01:18:43 (1732429123) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 01:18:44 (1732429124) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a4fe800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012a4fe800.ost_server_uuid in IDLE state after 0 sec oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a4fe800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012a4fe800.ost_server_uuid in FULL state after 0 sec oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a4fe800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012a4fe800.ost_server_uuid in IDLE state after 0 sec oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a4fe800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012a4fe800.ost_server_uuid in FULL state after 0 sec oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a4fe800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012a4fe800.ost_server_uuid in IDLE state after 0 sec oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a4fe800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012a4fe800.ost_server_uuid in FULL state after 0 sec PASS 34 (85s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 01:20:09 (1732429209) fail_loc=0x80000317 Race attempt 0 Wait for 19690 19839 for 60 sec... - open/close 2671 (time 1732429221.42 total 10.00 last 267.08) total: 4000 open/close in 12.45 seconds: 321.26 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.156@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre 192.168.203.156@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 01:21:13 (1732429273) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg356-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.300715 s, 174 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.292486 s, 179 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.286874 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7497728) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.285066 s, 184 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7495680) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.318221 s, 165 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7497728) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.284073 s, 185 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7495680) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.310259 s, 169 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.275764 s, 190 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.268093 s, 196 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.268374 s, 195 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7497728) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.264935 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7493632) after_dd(7442432) after(7493632) PASS 36 (224s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 01:24:57 (1732429497) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 total: 10000 create in 9.21 seconds: 1086.07 ops/second PASS 37 (11s) == sanityn test 39a: file mtime does not change after rename ========================================================== 01:25:08 (1732429508) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 01:25:10 (1732429510) repeat after cancel_lru_locks PASS 39b (2s) == sanityn test 39c: check truncate mtime update ================================================================================ 01:25:12 (1732429512) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 01:25:15 (1732429515) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 01:25:16 (1732429516) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 01:25:24 (1732429524) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 01:25:30 (1732429530) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 01:25:38 (1732429538) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 01:25:45 (1732429545) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 01:25:52 (1732429552) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 01:25:56 (1732429556) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 01:26:00 (1732429560) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 01:26:04 (1732429564) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 01:26:09 (1732429569) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 01:26:13 (1732429573) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 01:26:17 (1732429577) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 01:26:21 (1732429581) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 01:26:25 (1732429585) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (673s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 01:37:38 (1732430258) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 01:37:42 (1732430262) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 01:37:47 (1732430267) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 01:37:52 (1732430272) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 01:37:57 (1732430277) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 01:38:01 (1732430281) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 01:38:05 (1732430285) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 01:38:10 (1732430290) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 01:38:14 (1732430294) PASS 43a (15s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 01:38:29 (1732430309) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 01:38:34 (1732430314) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 01:38:38 (1732430318) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 01:38:42 (1732430322) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 01:38:47 (1732430327) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 01:38:51 (1732430331) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 01:38:56 (1732430336) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 01:39:00 (1732430340) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 01:39:01 (1732430341) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (34s) == sanityn test 43k: unlink vs create ==================== 01:39:35 (1732430375) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (865s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 01:54:00 (1732431240) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Sun Nov 24 01:54:07 EST 2024 PASS 44a (11s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 01:54:11 (1732431251) open: File exists Conflict PASS 44b (10s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 01:54:21 (1732431261) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (10s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 01:54:31 (1732431271) Conflict PASS 44d (9s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 01:54:40 (1732431280) Conflict PASS 44e (9s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 01:54:49 (1732431289) Conflict PASS 44f (11s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 01:55:00 (1732431300) Conflict PASS 44g (10s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 01:55:10 (1732431310) Conflict PASS 44h (10s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 01:55:20 (1732431320) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (2s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 01:55:22 (1732431322) PASS 45a (109s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 01:57:11 (1732431431) Conflict PASS 45b (12s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 01:57:23 (1732431443) Conflict PASS 45c (10s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 01:57:33 (1732431453) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (9s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 01:57:42 (1732431462) Conflict PASS 45e (10s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 01:57:52 (1732431472) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (10s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 01:58:02 (1732431482) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (10s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 01:58:12 (1732431492) Conflict PASS 45h (10s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 01:58:22 (1732431502) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (2s) == sanityn test 45j: read vs rename ====================== 01:58:25 (1732431505) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (943s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 02:14:08 (1732432448) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (10s) == sanityn test 46b: pdirops: link vs create ======================================================================== 02:14:19 (1732432459) open: File exists Conflict PASS 46b (9s) == sanityn test 46c: pdirops: link vs link ======================================================================== 02:14:27 (1732432467) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (10s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 02:14:37 (1732432477) Conflict PASS 46d (11s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 02:14:48 (1732432488) Conflict PASS 46e (12s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 02:15:00 (1732432500) Conflict PASS 46f (11s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 02:15:11 (1732432511) Conflict PASS 46g (10s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 02:15:21 (1732432521) Conflict PASS 46h (10s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 02:15:32 (1732432532) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (2s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 02:15:34 (1732432534) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (3s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 02:15:37 (1732432537) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (3s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 02:15:40 (1732432540) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 02:15:43 (1732432543) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (3s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 02:15:45 (1732432545) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (3s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 02:15:48 (1732432548) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (3s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 02:15:51 (1732432551) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (2s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 02:15:53 (1732432553) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.125922 s, 81.3 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (10s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 02:16:04 (1732432564) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.0239132 s, 6.6 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3518: kill: (32327) - No such process PASS 51a (8s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 02:16:11 (1732432571) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00806114 s, 127 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0510707 s, 20.5 MB/s 3145728 PASS 51b (17s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 02:16:28 (1732432588) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0517353 s, 19.8 kB/s PASS 51c (9s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 02:16:38 (1732432598) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336995 s, 31.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (7s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 02:16:44 (1732432604) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2448 0x990 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2448 0x990 0x240000400 read lease(1) has applied. PASS 51e (7s) == sanityn test 54: rename locking ======================= 02:16:51 (1732432611) llite.lustre-ffff88012a4fe800.xattr_cache=1 llite.lustre-ffff88012d0d7800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:16:52.000000000 -0500 Modify: 2024-11-24 02:16:52.000000000 -0500 Change: 2024-11-24 02:16:52.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:16:58.000000000 -0500 Modify: 2024-11-24 02:16:59.000000000 -0500 Change: 2024-11-24 02:16:59.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:17:04.000000000 -0500 Modify: 2024-11-24 02:17:04.000000000 -0500 Change: 2024-11-24 02:17:04.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:17:04.000000000 -0500 Modify: 2024-11-24 02:17:05.000000000 -0500 Change: 2024-11-24 02:17:05.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:17:11.000000000 -0500 Modify: 2024-11-24 02:17:11.000000000 -0500 Change: 2024-11-24 02:17:11.000000000 -0500 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:17:11.000000000 -0500 Modify: 2024-11-24 02:17:11.000000000 -0500 Change: 2024-11-24 02:17:11.000000000 -0500 Birth: - PASS 54 (30s) == sanityn test 55a: rename vs unlink target dir ========= 02:17:21 (1732432641) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (10s) == sanityn test 55b: rename vs unlink source dir ========= 02:17:32 (1732432652) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (10s) == sanityn test 55c: rename vs unlink orphan target dir == 02:17:43 (1732432663) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (16s) == sanityn test 55d: rename file vs link ================= 02:17:59 (1732432679) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (14s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 02:18:13 (1732432693) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1732432694 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 0.73541 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (88s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 02:19:41 (1732432781) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1732432784 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 1.60624 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (86s) == sanityn test 60: Verify data_version behaviour ======== 02:21:08 (1732432868) PASS 60 (6s) == sanityn test 70a: cd directory && rm directory ======== 02:21:13 (1732432873) PASS 70a (7s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 02:21:20 (1732432880) PASS 70b (6s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 02:21:26 (1732432886) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00888622 s, 7.4 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 02:21:29 (1732432889) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0233054 s, 2.8 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (2s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 02:21:31 (1732432891) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (2s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 02:21:34 (1732432894) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0246732 s, 2.7 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (2s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 02:21:36 (1732432896) llite.lustre-ffff88012a4fe800.xattr_cache=1 llite.lustre-ffff88012d0d7800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (5s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 02:21:41 (1732432901) llite.lustre-ffff88012a4fe800.xattr_cache=1 llite.lustre-ffff88012d0d7800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (4s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 02:21:45 (1732432905) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0260051 s, 39.4 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0227244 s, 45.1 kB/s 12507: get lock1 12506: get lock2 12506: done 12506: get lock1 12507: done 12507: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 12507: cannot get lock2: Resource deadlock avoided 12507: exit rc=0 12506: done 12506: exit rc=0 PASS 74 (9s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 02:21:54 (1732432914) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0436346 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203556 s, 51.5 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4107: 13150 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (11s) == sanityn test 76: Verify MDT open_files listing ======== 02:22:05 (1732432925) opening files: ........ closing files: ........ PASS 76 (52s) == sanityn test 77a: check FIFO NRS policy =============== 02:22:57 (1732432977) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.239139 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0627791 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0617355 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105191 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0987781 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132176 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143526 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103836 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13477 s, 7.8 MB/s 1048576 bytes (1.0 MB) copied, 0.114211 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191452 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.193757 s, 5.4 MB/s 1048576 bytes (1.0 MB) copied, 0.17934 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.211174 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.21248 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.211373 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.236594 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0711045 s, 14.7 MB/s , 0.0561187 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430293 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0710707 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294382 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529269 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267648 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290119 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0801076 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310829 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320437 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362264 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360406 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0426677 s, 24.6 MB/s , 0.0412939 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443603 s, 23.6 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (7s) == sanityn test 77b: check CRR-N NRS policy ============== 02:23:04 (1732432984) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.250559 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128425 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030546 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373174 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0980677 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106446 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0937902 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113367 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123321 s, 8.5 MB/s 1048576 bytes (1.0 MB) copied, 0.121973 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124387 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13715 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125407 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139013 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140422 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157315 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137766 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0503178 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0573927 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0649531 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662176 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0537311 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0726865 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0750627 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0822101 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0856591 s, 12.2 MB/s , 0.0830207 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0656559 s, 16.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0662148 s, 15.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0684108 s, 15.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0950951 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0710921 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.07111 s, 14.7 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.27779 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320263 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416396 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0969955 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0979388 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111462 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0810859 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102271 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123415 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121894 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139363 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144276 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140812 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123396 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142791 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161613 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185232 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365361 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0020384 s, 514 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495364 s, 21.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0816917 s, 12.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0205116 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425825 s, 24.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0920485 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0918583 s, 11.4 MB/s , 0.0568188 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589701 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582781 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492476 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0503503 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052462 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453073 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483979 s, 21.7 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (10s) == sanityn test 77c: check ORR NRS policy ================ 02:23:14 (1732432994) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.259942 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321275 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395897 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0609062 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0897605 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100283 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105987 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828202 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.074641 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121223 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113076 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0984836 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106943 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103978 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130695 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118378 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1563 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582417 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0583902 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02462 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440444 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0616002 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563282 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274746 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321073 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0591784 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.059085 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297604 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271537 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305716 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401377 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388414 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390665 s, 26.8 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.244421 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374996 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449361 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0959789 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105574 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126117 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130558 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127915 s, 8.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13858 s, 7.6 MB/s , 0.133377 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112425 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123252 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132523 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121208 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118674 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133599 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136635 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535245 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338847 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673256 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567828 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354971 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420361 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425059 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506043 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459512 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473677 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481844 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505292 s, 20.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0585585 s, 17.9 MB/s , 0.0594441 s, 17.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0594765 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603242 s, 17.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.193856 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354697 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158276 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368489 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0377731 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0287428 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0886825 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105278 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125172 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124339 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0909249 s, 11.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0935329 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0951809 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179585 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184731 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183133 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.206497 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0023847 s, 440 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027767 s, 37.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277255 s, 37.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0580093 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415718 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0583143 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697905 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496696 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048124 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319827 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346084 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0381757 s, 27.5 MB/s 1048576 bytes (1.0 MB) copied, 0.037397 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0655691 s, 16.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0412558 s, 25.4 MB/s , 0.0420608 s, 24.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (11s) == sanityn test 77d: check TRR nrs policy ================ 02:23:25 (1732433005) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.285183 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357878 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336829 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552357 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0879227 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0885496 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0870763 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0775589 s, 13.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0763404 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123706 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11006 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114728 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106382 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116805 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14427 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126624 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141393 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563588 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0611681 s, 17.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0671855 s, 15.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0572312 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0615051 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719306 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515808 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719974 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535377 s, 19.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0735035 s, 14.3 MB/s , 0.0724952 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582115 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.058319 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597042 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057298 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0595985 s, 17.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.218673 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311882 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292893 s, 35.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11675 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.099146 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106259 s, 9.9 MB/s 1048576 bytes (1.0 MB) copied, 0.119492 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112442 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145058 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15821 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142973 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163009 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149404 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162934 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188132 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168671 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.194145 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02896 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282533 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034182 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337044 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569335 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380161 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456456 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494265 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511316 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0522872 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535546 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571506 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0628512 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0648972 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0636987 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0593183 s, 17.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.215808 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238121 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164269 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404765 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010682 s, 98.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0377967 s, 27.7 MB/s , 0.016369 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142235 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131888 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140153 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150203 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138876 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135403 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185488 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186837 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191703 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.204318 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337019 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338147 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036005 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346642 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343845 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271535 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219827 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197346 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389445 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228531 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250226 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256594 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291649 s, 36.0 MB/s , 0.0288902 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326063 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364867 s, 28.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (10s) == sanityn test 77e: check TBF NID nrs policy ============ 02:23:35 (1732433015) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.56@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.283379 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362992 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304195 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444363 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0805729 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109176 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10316 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.119331 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0939529 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13221 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.136339 s, 7.7 MB/s , 0.143875 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127097 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158336 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156779 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158152 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156291 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0024523 s, 428 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337953 s, 31.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0417679 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.027031 s, 38.8 MB/s , 0.00625531 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129631 s, 80.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00514402 s, 204 MB/s , 0.084533 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00422498 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0706265 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319806 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0666291 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0585351 s, 17.9 MB/s , 0.0746824 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596232 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476705 s, 22.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.445043 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441926 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0624369 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166775 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148825 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191586 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192081 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.244583 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.202563 s, 5.2 MB/s 1048576 bytes (1.0 MB) copied, 0.248165 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.251438 s, 4.2 MB/s 1048576 bytes (1.0 MB) copied, 0.235677 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.21941 s, 4.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.214893 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.265864 s, 3.9 MB/s 1048576 bytes (1.0 MB) copied, 0.231897 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.234747 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417593 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597715 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0983036 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493627 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512112 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05991 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0654788 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673062 s, 15.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0718811 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0724803 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0797008 s, 13.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0847087 s, 12.4 MB/s , 0.0966396 s, 10.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0899266 s, 11.7 MB/s , 0.0666889 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10414 s, 10.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.270154 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268321 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363224 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405857 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473912 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0609656 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017989 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0644117 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0580673 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0766942 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160759 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143992 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155604 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181851 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156405 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150133 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181309 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674056 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00540928 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.072203 s, 14.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00241856 s, 434 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.084447 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0561478 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.064463 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363842 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0656705 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478868 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496938 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0682412 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519178 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538518 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.059929 s, 17.5 MB/s , 0.0594147 s, 17.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.23155 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441194 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393623 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105913 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0898652 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0702842 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0888173 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105765 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139477 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114595 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101336 s, 10.3 MB/s 1048576 bytes (1.0 MB) copied, 0.105789 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14505 s, 7.2 MB/s , 0.14406 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143742 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139437 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170716 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0680138 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00264185 s, 397 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552288 s, 19.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0507393 s, 20.7 MB/s , 0.0605218 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112159 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446958 s, 23.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0225436 s, 46.5 MB/s , 0.0380631 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390527 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022729 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251569 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345023 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043811 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343629 s, 30.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (19s) == sanityn test 77f: check TBF JobID nrs policy ========== 02:23:54 (1732433034) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.321237 s, 52.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316657 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172053 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490205 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271119 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303219 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407492 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.230997 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.24535 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.233513 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.249041 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.242603 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.256822 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.266769 s, 3.9 MB/s 1048576 bytes (1.0 MB) copied, 0.264186 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.294587 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.298199 s, 3.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268689 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370122 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403498 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633315 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371137 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406761 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495789 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597998 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513091 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527011 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535972 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0564604 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.056595 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594329 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638878 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645678 s, 16.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.243775 s, 68.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199759 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166092 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231841 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125338 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154274 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143713 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160765 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155543 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14908 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154396 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.207284 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.221186 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.220222 s, 4.8 MB/s , 0.22063 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.223709 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.26975 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565761 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452733 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463454 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469956 s, 22.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0739116 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05157 s, 20.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055586 s, 18.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0590989 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0585789 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0598798 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639034 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0657814 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674523 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0600597 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0761027 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0809849 s, 12.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.277538 s, 60.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0491333 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570086 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136739 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164359 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13712 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172984 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178098 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154944 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188401 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.209437 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.202069 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191144 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.21792 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.226369 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.249965 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.251249 s, 4.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390778 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433757 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484978 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395229 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633546 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455308 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589351 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563316 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645395 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0756607 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0773247 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0809193 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0720522 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0865176 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.075816 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0931298 s, 11.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.258301 s, 65.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264666 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304835 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0867406 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0973218 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0924574 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0974884 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0939398 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101615 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104124 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117808 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133802 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13083 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137473 s, 7.6 MB/s 1048576 bytes (1.0 MB) copied, 0.137538 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146179 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179212 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305248 s, 34.4 MB/s , 0.00259509 s, 404 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234021 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316538 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330092 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389678 s, 26.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417029 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338235 s, 31.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0367021 s, 28.6 MB/s , 0.0311067 s, 33.7 MB/s 1048576 bytes (1.0 MB) copied, 0.032639 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274315 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292066 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382297 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408497 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567505 s, 18.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (17s) == sanityn test 77g: Change TBF type directly ============ 02:24:11 (1732433051) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (7s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 02:24:18 (1732433058) oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg356-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg356-client: oleg356-server: ssh exited with exit code 2 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg356-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 12 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg356-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 12 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg356-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg356-client: oleg356-server: ssh exited with exit code 12 PASS 77h (8s) == sanityn test 77i: Change rank of TBF rule ============= 02:24:26 (1732433066) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (16s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 02:24:42 (1732433082) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.444588 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310737 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337278 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110475 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105811 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114121 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118599 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131705 s, 8.0 MB/s 1048576 bytes (1.0 MB) copied, 0.139989 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145296 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1529 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143788 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150698 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153767 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158385 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.201889 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197934 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00244688 s, 429 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0026909 s, 390 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659187 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0856805 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0626147 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0847131 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0909445 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0623382 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865724 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0732939 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504505 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105323 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408034 s, 25.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0875212 s, 12.0 MB/s , 0.0355969 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249361 s, 42.1 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.15478 s, 11.5 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6204 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (58s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 02:25:41 (1732433141) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.26192 s, 13.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388008 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0590367 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0976511 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0686037 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0862249 s, 12.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0774076 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128855 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112789 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147229 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14701 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179031 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.219973 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.247171 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.245521 s, 4.3 MB/s 1048576 bytes (1.0 MB) copied, 0.2309 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.252497 s, 4.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506465 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432888 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0556642 s, 18.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0474124 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473382 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633138 s, 16.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0680286 s, 15.4 MB/s , 0.0601416 s, 17.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0694393 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589405 s, 17.8 MB/s 1048576 bytes (1.0 MB) copied, 0.063612 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0628063 s, 16.7 MB/s 1048576 bytes (1.0 MB) copied, 0.072251 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0682799 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0770418 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107828 s, 9.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9253 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9195 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.29138 s, 13.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659739 s, 15.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0691414 s, 15.2 MB/s , 0.0682768 s, 15.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0737275 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104557 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108178 s, 9.7 MB/s , 0.122888 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107314 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105838 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110249 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14226 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143726 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13335 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151888 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187675 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.204639 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051002 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321456 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325765 s, 32.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0357739 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439253 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451191 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0394843 s, 26.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.039581 s, 26.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0391602 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0549407 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0424256 s, 24.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0509261 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434172 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467179 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049938 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0632525 s, 16.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9673 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.93 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (141s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 02:28:01 (1732433281) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.27613 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0593141 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.054797 s, 19.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0882288 s, 11.9 MB/s , 0.0845482 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0789856 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106697 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125138 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120238 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133921 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143797 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144603 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169642 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183343 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192511 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.202039 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.209418 s, 5.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376483 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388809 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383341 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414469 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414417 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447175 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465255 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468812 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486969 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468721 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494642 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0531888 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545076 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541699 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538146 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592869 s, 17.7 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8998 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9512 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg356-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.25238 s, 13.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396857 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0585035 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0851471 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0850274 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0996424 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.119385 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103416 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102802 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109235 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156632 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152182 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160203 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177495 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180798 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190017 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.220756 s, 4.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.026845 s, 39.1 MB/s , 0.0264876 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372283 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292688 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391964 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316815 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318954 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425811 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404198 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341456 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039715 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507221 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439742 s, 23.8 MB/s , 0.0432772 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0551127 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489509 s, 21.4 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8782 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9369 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (137s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 02:30:18 (1732433418) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.253126 s, 66.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140891 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252499 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228315 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405226 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387025 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426092 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528042 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395938 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541178 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516816 s, 20.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0540914 s, 19.4 MB/s , 0.0511396 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534805 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534346 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527766 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0694195 s, 15.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112229 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926255 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144674 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126805 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121892 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126987 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110538 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143552 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152029 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133935 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158551 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160616 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156003 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156576 s, 67.0 MB/s , 0.0148883 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186629 s, 56.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98106 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.75179 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.56@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.56@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0952234 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837013 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136418 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0254315 s, 41.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0240269 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224792 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260424 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150124 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278392 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00740524 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233665 s, 44.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.214181 s, 4.9 MB/s , 0.220564 s, 4.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.257432 s, 4.1 MB/s , 0.255575 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314419 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.315995 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00772367 s, 136 MB/s , 0.00768408 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880146 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804568 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00413155 s, 254 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928567 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993098 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919349 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869013 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0118903 s, 88.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0144035 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101684 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982766 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128128 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109974 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138552 s, 75.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92496 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73045 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.56@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.121728 s, 138 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102864 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169143 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335888 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380709 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373643 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387509 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437766 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414255 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039368 s, 26.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388157 s, 27.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0480095 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048205 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0529389 s, 19.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0598711 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592944 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582418 s, 18.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00783669 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787545 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708329 s, 148 MB/s 1048576 bytes (1.0 MB) copied, 0.00885323 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108126 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124771 s, 84.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0113599 s, 92.3 MB/s , 0.0146323 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00623906 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00169962 s, 617 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0096214 s, 109 MB/s , 0.0082346 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129264 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00178867 s, 586 MB/s 1048576 bytes (1.0 MB) copied, 0.00160797 s, 652 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00144338 s, 726 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97863 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89881 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.152317 s, 110 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124429 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142145 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958034 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399419 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446868 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433629 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432414 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353773 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475327 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044059 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481952 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479712 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507519 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490109 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0960091 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101445 s, 10.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118975 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136688 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0130657 s, 80.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0092545 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196164 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104395 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189787 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154264 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156621 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162488 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125183 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124904 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0145855 s, 71.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0137678 s, 76.2 MB/s , 0.0100842 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128557 s, 81.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.87439 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7572 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97752 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.75646 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.142848 s, 117 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104012 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184759 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166961 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326814 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346462 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368996 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033166 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337499 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10108 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11579 s, 9.1 MB/s 1048576 bytes (1.0 MB) copied, 0.119238 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121848 s, 8.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.138206 s, 7.6 MB/s , 0.139504 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137961 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144008 s, 7.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00942051 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160405 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121529 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201094 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112907 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138375 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143149 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121739 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013397 s, 78.3 MB/s , 0.0134803 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142987 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150034 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160237 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164383 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164516 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017499 s, 59.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6867 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4558 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106705 s, 157 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00753228 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00776667 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786909 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119237 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191213 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175275 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030035 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299896 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028829 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02297 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286242 s, 36.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126699 s, 82.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0187655 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014513 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129459 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108537 s, 96.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00911363 s, 115 MB/s , 0.00874433 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957975 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937515 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109296 s, 95.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0121353 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00159022 s, 659 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112734 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115785 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996521 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105825 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105753 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107671 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108075 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126125 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125485 s, 83.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94038 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73801 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.134388 s, 125 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106227 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921029 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142617 s, 73.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287903 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265323 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308176 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308606 s, 34.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0328127 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359977 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267931 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322385 s, 32.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0391654 s, 26.8 MB/s , 0.0285605 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439148 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105854 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108501 s, 9.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108641 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00187912 s, 558 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265252 s, 39.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0232278 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218738 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00177295 s, 591 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00333481 s, 314 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191066 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028309 s, 37.0 MB/s , 0.0282885 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195194 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169196 s, 62.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0210485 s, 49.8 MB/s , 0.0297531 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769589 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187624 s, 55.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.84998 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73678 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82402 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76759 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (346s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 02:36:04 (1732433764) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 02:36:07 (1732433767) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg356-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.57564 s, 293 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg356-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.015 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 02:36:57 (1732433817) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0990557 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105132 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110738 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019281 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281414 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331809 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337697 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355119 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429924 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440282 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436295 s, 24.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529592 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492053 s, 21.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0474959 s, 22.1 MB/s , 0.0495849 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535766 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050523 s, 20.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a4fe800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a4fe800.lock_unused_count=5 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00874406 s, 120 MB/s , 0.00789993 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00884673 s, 119 MB/s , 0.00615585 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971885 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117904 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00840681 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00855743 s, 123 MB/s , 0.00864247 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940374 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971117 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990969 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105655 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107886 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010886 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120985 s, 86.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg356-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93019 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.968 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.234515 s, 71.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128694 s, 81.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00779415 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285016 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342425 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299805 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300349 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447579 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438546 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439417 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361123 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416919 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040596 s, 25.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.272563 s, 3.8 MB/s , 0.276679 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.326669 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.32776 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197673 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199441 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019862 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194392 s, 53.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0183541 s, 57.1 MB/s , 0.0184281 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196162 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207014 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202343 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248992 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207798 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209423 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225054 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.022779 s, 46.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0228245 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022764 s, 46.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg356-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg356-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9292 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97057 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (61s) == sanityn test 77o: Changing rank should not panic ====== 02:37:58 (1732433878) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 02:38:02 (1732433882) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 02:38:28 (1732433908) mds.MDS.mdt.nrs_policies=tbf oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg356-client: oleg356-server: ssh exited with exit code 36 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 oleg356-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg356-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg356-client: oleg356-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 02:38:37 (1732433917) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 9688 kill md_thread xargs PASS 77r (37s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 02:39:14 (1732433954) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 02:39:16 (1732433956) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506510 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:39:17.000000000 -0500 Modify: 2024-11-24 02:39:17.000000000 -0500 Change: 2024-11-24 02:39:17.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 02:39:28 (1732433968) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 02:39:29 (1732433969) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 02:39:30 (1732433970) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (0s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 02:39:30 (1732433970) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 02:39:31 (1732433971) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 02:39:32 (1732433972) total: 2500 open/close in 5.19 seconds: 481.41 ops/second snapshot_time 1732433992.089893933 secs.nsecs start_time 1732433973.553984019 secs.nsecs elapsed_time 18.535909914 secs.nsecs open 2505 samples [usecs] 69 1529 674753 192961923 close 2504 samples [usecs] 55 757 209791 19838575 mknod 2501 samples [usecs] 154 1493 595990 151842848 unlink 2500 samples [usecs] 273 8925 1086969 848860243 mkdir 2 samples [usecs] 623 707 1330 887978 rmdir 2 samples [usecs] 1867 2919 4786 12006250 rename 2500 samples [usecs] 224 9857 2338737 3233359879 getattr 3760 samples [usecs] 7 517 61603 1370863 parallel_rename_file 2500 samples [usecs] 224 9857 2338737 3233359879 crossdir_rename 2500 samples [usecs] 224 9857 2338737 3233359879 PASS 81d (21s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 02:39:53 (1732433993) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6978 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6978 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 02:39:54 (1732433994) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 02:39:55 (1732433995) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 85: Lustre API root cache race =========== 02:40:02 (1732434002) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1732434002 Finishing test test1 at 1732434003 PASS 85 (2s) == sanityn test 90: open/create and unlink striped directory ========================================================== 02:40:04 (1732434004) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 02:40:05 (1732434005) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 02:40:06 (1732434006) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 02:40:07 (1732434007) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5806 0x16ae 0x280000400 0 5896 0x1708 0x240000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5897 0x1709 0x240000400 1 5807 0x16af 0x280000400 rc1=0 and rc2=0 PASS 93 (7s) == sanityn test 94: signal vs CP callback race =========== 02:40:14 (1732434014) osc.lustre-OST0000-osc-ffff88012a4fe800.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012d0d7800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012a4fe800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012d0d7800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0233366 s, 351 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99896 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5743: 29065 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0128869 s, 318 kB/s osc.lustre-OST0000-osc-ffff88012a4fe800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012d0d7800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a4fe800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012d0d7800.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 02:40:24 (1732434024) llite.lustre-ffff88012a4fe800.fast_read=0 llite.lustre-ffff88012d0d7800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0150334 s, 139 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff88012a4fe800.fast_read=1 llite.lustre-ffff88012d0d7800.fast_read=1 PASS 95a (19s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 02:40:43 (1732434043) llite.lustre-ffff88012a4fe800.fast_read=0 llite.lustre-ffff88012d0d7800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00152674 s, 8.0 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff88012a4fe800.fast_read=1 llite.lustre-ffff88012d0d7800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 02:40:51 (1732434051) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (0s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 02:40:51 (1732434051) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00189156 s, 2.2 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 02:40:53 (1732434053) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0223586 s, 93.8 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 02:40:55 (1732434055) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.019912 s, 105 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 02:40:57 (1732434057) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00152229 s, 7.9 MB/s PASS 100e (1s) == sanityn test 101a: Discard DoM data on unlink ========= 02:40:58 (1732434058) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000664622 s, 6.2 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 02:41:00 (1732434060) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000506515 s, 8.1 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 02:41:02 (1732434062) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000623207 s, 6.6 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 02:41:05 (1732434065) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1732434066 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1732434068 Finishing test test23 at 1732434068 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 02:41:12 (1732434072) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (0s) == sanityn test 105: Glimpse and lock cancel race ======== 02:41:12 (1732434072) fail_loc=0x416 PASS 105 (22s) == sanityn test 106a: Verify the btime via statx() ======= 02:41:35 (1732434095) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 02:41:35 (1732434095) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 02:41:36 (1732434096) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 02:41:37 (1732434097) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0591516 s, 177 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5812 0x16b4 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 02:41:41 (1732434101) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0608207 s, 172 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5905 0x1711 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6978 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (7s) == sanityn test 108a: lseek: parallel updates ============ 02:41:48 (1732434108) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00151298 s, 6.6 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.1764 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 02:41:54 (1732434114) 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 45 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 46 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg356-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 50 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre 192.168.203.156@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg356-server' oleg356-server: oleg356-server.virtnet: executing load_modules_local oleg356-server: Loading modules from /home/green/git/lustre-release/lustre oleg356-server: detected 4 online CPUs by sysfs oleg356-server: Force libcfs to create 2 CPU partitions Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre Starting client: oleg356-client.virtnet: -o user_xattr,flock 192.168.203.156@tcp:/lustre /mnt/lustre2 PASS 109 (391s) == sanityn test 110: do not grant another lock on resend ========================================================== 02:48:25 (1732434505) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-11-24 02:48:26.000000000 -0500 Modify: 2024-11-24 02:48:26.000000000 -0500 Change: 2024-11-24 02:48:26.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg356-client.virtnet oleg356-client.virtnet PASS 110 (65s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 02:49:30 (1732434570) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 02:49:31 (1732434571) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 02:49:32 (1732434572) lustre-OST0000-osc-ffff8800a8b7f800 active. lustre-OST0001-osc-ffff8800a8b7f800 active. lustre-OST0000-osc-ffff8800a8b7f800 active. lustre-OST0001-osc-ffff8800a8b7f800 active. MGC192.168.203.156@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 02:49:34 (1732434574) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (0s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 02:49:34 (1732434574) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 02:49:35 (1732434575) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 02:49:36 (1732434576) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=22 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.0196 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.0269 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.0315 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.116 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.121 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1237 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.1228 s, 0.2 kB/s , 20.1197 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1213 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.0988 s, 0.1 kB/s , 40.1088 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1187 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.1325 s, 0.1 kB/s , 40.0966 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1429 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1366 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1883 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.216 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.1123 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.1333 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.1334 s, 0.1 kB/s , 60.1593 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.1403 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.2233 s, 0.1 kB/s , 60.2066 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3037 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3086 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.1389 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.201 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.1977 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.2298 s, 0.1 kB/s , 80.2328 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2394 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2567 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3759 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4261 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.217 s, 0.0 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 100.264 s, 0.0 kB/s , 100.261 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 100.324 s, 0.0 kB/s 4096 bytes (4.1 kB) copied, 100.321 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.34 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.335 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.489 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg356-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a8b7f800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a8b7f800.ost_server_uuid in FULL state after 0 sec PASS 200 (106s) cleanup: ====================================================== == sanityn test complete, duration 6015 sec ============== 02:51:22 (1732434682) === sanityn: start cleanup 02:51:23 (1732434683) === Stopping clients: oleg356-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg356-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 02:52:01 (1732434721) ===