-----============= acceptance-small: sanityn ============----- Fri Dec 13 13:42:02 EST 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg204-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 13:42:07 (1734115327) === Starting client oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Started clients oleg204-client.virtnet: 192.168.202.104@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg204-client.virtnet: executing check_config_client /mnt/lustre oleg204-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg204-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b54d5800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6d7d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b54d5800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6d7d000.idle_timeout=debug disable quota as required oleg204-server: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 13:42:14 (1734115334) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162557 s, 64.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f6987] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 13:42:15 (1734115335) PASS 0 (3s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 13:42:18 (1734115338) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 13:42:20 (1734115340) -rw-r--r-- 1 root root 0 Dec 13 13:42 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 13:42:21 (1734115341) -rw-r--r-- 1 root root 0 Dec 13 13:42 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 13:42:23 (1734115343) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 13:42:25 (1734115345) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 13:42:27 (1734115347) total 1 -rwxrwxrwx 1 root root 0 Dec 13 13:42 f2a -rwxrwxrwx 1 root root 0 Dec 13 13:42 f2b total 1 -rwxrwxrwx 1 root root 0 Dec 13 13:42 f2a -rwxrwxrwx 1 root root 0 Dec 13 13:42 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00621313 s, 82.4 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 13:42:29 (1734115349) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 13:42:30 (1734115350) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.119019 s, 17.6 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 13:42:32 (1734115352) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 13:42:34 (1734115354) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 13:42:38 (1734115358) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 13:42:40 (1734115360) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 13:42:43 (1734115363) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 13:42:45 (1734115365) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 13:42:47 (1734115367) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 13:42:50 (1734115370) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158849 s, 66.0 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.015529 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 13:42:52 (1734115372) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00431806 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00527818 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00606067 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00420193 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00733125 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00619323 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00574673 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0057821 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00418152 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00351733 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0079346 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00477572 s, 0.2 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 13:42:55 (1734115375) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00501808 s, 612 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0173852 s, 236 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000241506 s, 12.7 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 13:42:57 (1734115377) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 13:42:59 (1734115379) warning: '-runtime' deprecated, use '-t runtime' instead - link 1073 (time 1734115390.42 total 10.01 last 107.23) - link 2222 (time 1734115400.43 total 20.01 last 114.86) - link 3529 (time 1734115410.43 total 30.01 last 130.65) - link 5058 (time 1734115420.43 total 40.01 last 152.88) - link 6915 (time 1734115430.44 total 50.02 last 185.62) - link 8628 (time 1734115440.44 total 60.02 last 171.28) - link 10000 (time 1734115448.55 total 68.13 last 169.20) - link 11850 (time 1734115458.55 total 78.13 last 184.92) - link 13741 (time 1734115468.55 total 88.14 last 189.02) - link 15463 (time 1734115478.56 total 98.14 last 172.11) total: 15754 link in 100.00 seconds: 157.54 ops/second using seed 4103547904 running for 100 seconds - stat 10000 (time 1734115388 ; total 7 ; last 7) - stat 20000 (time 1734115394 ; total 13 ; last 6) - stat 30000 (time 1734115401 ; total 20 ; last 7) - stat 40000 (time 1734115407 ; total 26 ; last 6) - stat 50000 (time 1734115413 ; total 32 ; last 6) - stat 60000 (time 1734115419 ; total 38 ; last 6) - stat 70000 (time 1734115424 ; total 43 ; last 5) - stat 80000 (time 1734115430 ; total 49 ; last 6) - stat 86430 (time 1734115441 ; total 60 ; last 11) - stat 90000 (time 1734115447 ; total 66 ; last 6) - stat 96170 (time 1734115458 ; total 77 ; last 11) - stat 100000 (time 1734115464 ; total 83 ; last 6) - stat 106476 (time 1734115475 ; total 94 ; last 11) total: 109902 stats in 100 seconds: 1099.020020 stats/second - unlinked 0 (time 1734115431 ; total 0 ; last 0) - unlinked 10000 (time 1734115502 ; total 71 ; last 71) unlink(/mnt/lustre2/lockdir/lockfile15754) error: No such file or directory total: 15753 unlinks in 100 seconds: 157.529999 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22002) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22005) - No such process PASS 12 (154s) == sanityn test 13: test directory page revocation ======= 13:45:33 (1734115533) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 13:45:35 (1734115535) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4860: 23406 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 13:45:38 (1734115538) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4860: 24040 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 13:45:40 (1734115540) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24674 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 13:45:42 (1734115542) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25310 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 13:45:43 (1734115543) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 25944 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 13:45:46 (1734115546) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (13s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 13:45:59 (1734115559) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.184158 s, 45.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.181705 s, 46.2 MB/s Chance of close/open is 1 in 50 Seed set to 5560 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1734115560.878658 mapwrite 0x27b471 thru 0x285f69 (0xaaf9 bytes) 000003[0] 1734115560.956964 mapread 0x15f530 thru 0x16d900 (0xe3d1 bytes) 000004[1] 1734115560.972947 read 0x276730 thru 0x2827d3 (0xc0a4 bytes) 000005[1] 1734115560.990222 write 0x1d4440 thru 0x1ded4b (0xa90c bytes) 000008[0] 1734115561.013177 read 0x0a501d thru 0x0afcb9 (0xac9d bytes) 000009[1] 1734115561.034737 write 0x5340be thru 0x5397fa (0x573d bytes) 000012[1] 1734115561.060089 read 0x401e22 thru 0x40e2d6 (0xc4b5 bytes) 000013[0] 1734115561.064670 mapread 0x3e8490 thru 0x3f2696 (0xa207 bytes) 000017[0] 1734115561.138776 write 0x6b0d83 thru 0x6bfa0b (0xec89 bytes) 000019[1] 1734115561.163011 mapwrite 0x16c409 thru 0x178f6d (0xcb65 bytes) truncating to largest ever: 0x5f1db4 000021[1] 1734115561.232773 trunc from 0x6bfa0c to 0x5f1db3 (0xcdc58 bytes) 000022[1] 1734115561.260249 read 0x323ee3 thru 0x327945 (0x3a63 bytes) 000025[0] 1734115561.314329 mapread 0x1674af thru 0x16c369 (0x4ebb bytes) 000026[1] 1734115561.339721 trunc from 0x5f1db4 to 0x4b84a8 (0x13990b bytes) 000029[1] 1734115561.373424 mapread 0x35a836 thru 0x3678ca (0xd095 bytes) 000031[0] 1734115561.413430 write 0x6b8b6a thru 0x6c3063 (0xa4fa bytes) 000033[1] 1734115561.433678 mapread 0x4cce82 thru 0x4dcb45 (0xfcc4 bytes) 000036[1] 1734115561.492612 mapwrite 0x870b07 thru 0x87cae9 (0xbfe3 bytes) truncating to largest ever: 0x687008 000038[0] 1734115561.565783 trunc from 0x87caea to 0x687007 (0x1f5ae2 bytes) 000042[1] 1734115561.607386 mapwrite 0x2e97e2 thru 0x2f6d88 (0xd5a7 bytes) 000043[0] 1734115561.682779 read 0x4fd578 thru 0x4fd76f (0x1f8 bytes) 000044[0] 1734115561.693297 write 0x30aba4 thru 0x310c7c (0x60d9 bytes) 000045[1] 1734115561.734122 trunc from 0x687008 to 0x61597d (0x7168a bytes) 000046[1] 1734115561.761554 mapread 0x513fac thru 0x51fe50 (0xbea5 bytes) 000047[1] 1734115561.790070 write 0x871290 thru 0x87245a (0x11cb bytes) 000048[1] 1734115561.791197 mapread 0x3e2454 thru 0x3e8ebe (0x6a6b bytes) 000050[1] 1734115561.822001 trunc from 0x87245b to 0x2c31f5 (0x5af265 bytes) 000053[0] 1734115561.850474 trunc from 0x2c31f6 to 0x2c5aba (0x28c5 bytes) 000055[0] 1734115561.877126 read 0x1ef5d7 thru 0x1f2a8f (0x34b9 bytes) 000060[0] 1734115561.920769 mapread 0x29fe38 thru 0x2af6ae (0xf877 bytes) 000062[1] 1734115561.930736 mapread 0x2bad60 thru 0x2bc201 (0x14a2 bytes) 000063[0] 1734115561.955929 mapwrite 0x09cf13 thru 0x0aa5f0 (0xd6de bytes) 000064[0] 1734115562.036563 mapwrite 0x6ff114 thru 0x6ff921 (0x80e bytes) 000066[1] 1734115562.088213 mapread 0x53eae1 thru 0x54abf2 (0xc112 bytes) 000077[1] 1734115562.182438 mapread 0x530810 thru 0x53645c (0x5c4d bytes) 000078[0] 1734115562.200792 mapread 0x692c7e thru 0x69ee71 (0xc1f4 bytes) 000079[0] 1734115562.243446 mapwrite 0x3c4fbc thru 0x3cd277 (0x82bc bytes) 000083[0] 1734115562.289073 read 0x20c091 thru 0x20ca1a (0x98a bytes) 000084[1] 1734115562.313916 mapread 0x05ddaf thru 0x06249a (0x46ec bytes) 000086[0] 1734115562.339375 mapwrite 0x5e821e thru 0x5ee1c8 (0x5fab bytes) 000087[1] 1734115562.429805 write 0x046422 thru 0x04eb63 (0x8742 bytes) truncating to largest ever: 0x8ab638 000089[1] 1734115562.453954 trunc from 0x6ff922 to 0x8ab637 (0x1abd16 bytes) 000095[0] 1734115562.495442 mapwrite 0x7ea5af thru 0x7f7491 (0xcee3 bytes) 000096[0] 1734115562.559468 read 0x5d76c7 thru 0x5e05d6 (0x8f10 bytes) 000099[0] 1734115562.612237 write 0x9cfdc6 thru 0x9dafd3 (0xb20e bytes) 000100[1] 1734115562.622252 mapread 0x1a59d4 thru 0x1acff4 (0x7621 bytes) 000104[0] 1734115562.696550 mapwrite 0x8eb8c9 thru 0x8f7515 (0xbc4d bytes) 000108[1] 1734115562.826307 write 0x49b7c5 thru 0x49f5fa (0x3e36 bytes) 000109[0] 1734115562.843834 write 0x410e33 thru 0x4176d5 (0x68a3 bytes) 000111[0] 1734115562.857383 read 0x786c92 thru 0x793b7c (0xceeb bytes) 000112[0] 1734115562.895067 trunc from 0x9dafd4 to 0x7119e6 (0x2c95ed bytes) 000113[0] 1734115562.909149 read 0x57007b thru 0x5719a2 (0x1928 bytes) 000115[0] 1734115562.909770 read 0x53107f thru 0x53a202 (0x9184 bytes) 000116[1] 1734115562.921116 write 0x120f5e thru 0x1220f9 (0x119c bytes) 000117[0] 1734115562.939647 read 0x5e4355 thru 0x5e4aee (0x79a bytes) 000119[0] 1734115562.953420 mapwrite 0x327641 thru 0x32ed6b (0x772b bytes) 000120[1] 1734115563.036207 mapwrite 0x1cc250 thru 0x1daaa0 (0xe851 bytes) 000121[0] 1734115563.121575 trunc from 0x7119e7 to 0x4af3a1 (0x262645 bytes) 000124[0] 1734115563.158847 read 0x0a5594 thru 0x0a80dd (0x2b4a bytes) 000127[0] 1734115563.207729 trunc from 0x4af3a2 to 0x0830a2 (0x42c2ff bytes) 000129[0] 1734115563.243599 mapwrite 0x6d693a thru 0x6e22a8 (0xb96f bytes) 000130[1] 1734115563.323731 write 0x8e2ebd thru 0x8ee0c4 (0xb208 bytes) 000133[0] 1734115563.363257 mapread 0x3386f1 thru 0x342a94 (0xa3a4 bytes) 000138[1] 1734115563.439872 write 0x38835b thru 0x38a09b (0x1d41 bytes) 000139[0] 1734115563.459108 mapread 0x4c9818 thru 0x4d85bd (0xeda6 bytes) 000141[1] 1734115563.478646 mapwrite 0x6406b4 thru 0x644bf7 (0x4544 bytes) 000143[1] 1734115563.536988 read 0x82445a thru 0x824ad1 (0x678 bytes) 000144[0] 1734115563.567416 read 0x623436 thru 0x629cba (0x6885 bytes) 000145[1] 1734115563.595209 mapwrite 0x8fbcc1 thru 0x905a6a (0x9daa bytes) 000146[1] 1734115563.677888 mapread 0x4e36bc thru 0x4eddb6 (0xa6fb bytes) 000147[0] 1734115563.698841 mapread 0x0e7127 thru 0x0ec979 (0x5853 bytes) 000154[0] 1734115563.788626 mapread 0x29e811 thru 0x2aa611 (0xbe01 bytes) 000157[1] 1734115563.801318 read 0x792409 thru 0x794b5b (0x2753 bytes) 000159[1] 1734115563.815028 mapread 0x0e2160 thru 0x0eff84 (0xde25 bytes) 000165[1] 1734115563.872121 read 0x41afe9 thru 0x420183 (0x519b bytes) 000166[1] 1734115563.908009 write 0x8e1be6 thru 0x8e28a3 (0xcbe bytes) 000167[0] 1734115563.935640 mapread 0x26320d thru 0x26c334 (0x9128 bytes) 000168[1] 1734115563.974561 mapread 0x23b22a thru 0x2405ad (0x5384 bytes) 000169[1] 1734115563.997303 write 0x1b46f4 thru 0x1ba319 (0x5c26 bytes) 000176[1] 1734115564.045808 trunc from 0x905a6b to 0x23f633 (0x6c6437 bytes) 000179[0] 1734115564.073104 read 0x12c2f4 thru 0x135080 (0x8d8d bytes) 000181[0] 1734115564.138384 trunc from 0x23f634 to 0x68336d (0x443d3a bytes) 000182[1] 1734115564.164932 read 0x64add9 thru 0x64fc45 (0x4e6d bytes) 000183[0] 1734115564.186037 mapwrite 0x0a143d thru 0x0a4b16 (0x36da bytes) 000184[1] 1734115564.271556 mapwrite 0x88c300 thru 0x8904c5 (0x41c6 bytes) 000185[1] 1734115564.396209 trunc from 0x8904c6 to 0x397506 (0x4f8fbf bytes) 000187[0] 1734115564.415923 trunc from 0x397507 to 0x4d7de9 (0x1408e3 bytes) 000188[0] 1734115564.435359 mapread 0x482d01 thru 0x4847f2 (0x1af2 bytes) 000189[0] 1734115564.441411 mapwrite 0x508d37 thru 0x510d02 (0x7fcc bytes) 000190[1] 1734115564.497078 write 0x486489 thru 0x4882de (0x1e56 bytes) 000191[1] 1734115564.510350 read 0x2b483d thru 0x2bcf6b (0x872f bytes) 000193[0] 1734115564.570897 trunc from 0x510d03 to 0x2d2cf7 (0x23e00b bytes) 000194[0] 1734115564.596650 mapwrite 0x61cf5f thru 0x62868c (0xb72e bytes) 000195[0] 1734115564.675584 read 0x1b5503 thru 0x1bd04e (0x7b4c bytes) 000196[0] 1734115564.726966 mapwrite 0x51bb71 thru 0x529c7d (0xe10d bytes) 000199[0] 1734115564.771611 mapwrite 0x01a0b5 thru 0x028cb2 (0xebfe bytes) 000201[0] 1734115564.823407 write 0x9740b9 thru 0x9832a3 (0xf1eb bytes) 000203[0] 1734115564.843796 read 0x6ff359 thru 0x70967d (0xa325 bytes) 000205[1] 1734115564.866639 write 0x61a758 thru 0x62a5bb (0xfe64 bytes) 000207[1] 1734115564.901878 mapwrite 0x8a603e thru 0x8a7cfa (0x1cbd bytes) 000209[1] 1734115564.967596 mapwrite 0x1bf1d1 thru 0x1c34f1 (0x4321 bytes) 000212[0] 1734115565.023377 mapwrite 0x613ad2 thru 0x622f12 (0xf441 bytes) 000213[0] 1734115565.107583 read 0x5d95fb thru 0x5e77eb (0xe1f1 bytes) 000214[0] 1734115565.140351 trunc from 0x9832a4 to 0x3a2954 (0x5e094f bytes) 000217[0] 1734115565.157300 trunc from 0x3a2955 to 0x3ae8ba (0xbf66 bytes) 000218[0] 1734115565.178582 write 0x51e147 thru 0x5239d3 (0x588d bytes) 000222[1] 1734115565.199685 mapwrite 0x39248c thru 0x3a10ae (0xec23 bytes) 000224[0] 1734115565.298680 write 0x539fba thru 0x543810 (0x9857 bytes) 000228[0] 1734115565.334009 trunc from 0x543811 to 0x26bd67 (0x2d7aa9 bytes) 000229[0] 1734115565.352758 mapwrite 0x17c361 thru 0x189440 (0xd0e0 bytes) 000231[0] 1734115565.405202 mapwrite 0x5db770 thru 0x5e576d (0x9ffe bytes) 000236[0] 1734115565.472599 trunc from 0x5e576e to 0x7d13ba (0x1ebc4d bytes) 000240[0] 1734115565.500286 mapread 0x111534 thru 0x113dcf (0x289c bytes) 000242[1] 1734115565.520159 mapwrite 0x73ae7f thru 0x747b56 (0xccd8 bytes) 000243[1] 1734115565.570989 mapwrite 0x3ff986 thru 0x409395 (0x9a10 bytes) 000245[1] 1734115565.628720 read 0x0d67f9 thru 0x0db84b (0x5053 bytes) 000246[0] 1734115565.651600 mapwrite 0x092fae thru 0x094b61 (0x1bb4 bytes) truncating to largest ever: 0x91acff 000248[1] 1734115565.716182 trunc from 0x7d13bb to 0x91acfe (0x149944 bytes) 000250[0] 1734115565.754807 mapwrite 0x8c17ba thru 0x8cbdc2 (0xa609 bytes) 000252[0] 1734115565.821145 write 0x804b79 thru 0x80d86b (0x8cf3 bytes) 000253[1] 1734115565.835195 mapread 0x21d5a1 thru 0x2241b4 (0x6c14 bytes) 000254[1] 1734115565.876902 mapwrite 0x85689d thru 0x866845 (0xffa9 bytes) 000255[0] 1734115565.972327 mapread 0x743721 thru 0x74bb2a (0x840a bytes) 000268[1] 1734115566.062054 mapwrite 0x5b30bf thru 0x5bbb17 (0x8a59 bytes) 000269[0] 1734115566.121290 trunc from 0x91acff to 0x7b4ad0 (0x16622e bytes) 000273[0] 1734115566.158738 read 0x30fab8 thru 0x31b2ab (0xb7f4 bytes) 000275[0] 1734115566.175429 read 0x481a88 thru 0x48c90a (0xae83 bytes) truncating to largest ever: 0x937dfc 000276[1] 1734115566.230372 trunc from 0x7b4ad1 to 0x937dfb (0x18332b bytes) 000278[1] 1734115566.275177 trunc from 0x937dfc to 0x893c81 (0xa417a bytes) 000279[1] 1734115566.298135 mapwrite 0x362cee thru 0x36a76e (0x7a81 bytes) 000281[0] 1734115566.371343 write 0x7e8b6a thru 0x7f119a (0x8631 bytes) 000282[1] 1734115566.387732 trunc from 0x893c82 to 0x663cc5 (0x22ffbc bytes) 000287[1] 1734115566.433047 trunc from 0x663cc6 to 0x3e2e7b (0x280e4a bytes) 000288[0] 1734115566.456552 write 0x615d7c thru 0x61adb8 (0x503d bytes) 000289[0] 1734115566.471230 write 0x7fa10c thru 0x7fb5fe (0x14f3 bytes) 000293[0] 1734115566.503633 write 0x26d7bc thru 0x26eb22 (0x1367 bytes) 000295[0] 1734115566.520272 mapwrite 0x3be2e1 thru 0x3bed0e (0xa2e bytes) 000296[1] 1734115566.556365 read 0x3a1243 thru 0x3a81b6 (0x6f74 bytes) 000297[0] 1734115566.606693 write 0x5b2287 thru 0x5beb5a (0xc8d4 bytes) 000298[0] 1734115566.613147 mapread 0x7a6a47 thru 0x7aa81a (0x3dd4 bytes) 000301[1] 1734115566.633312 trunc from 0x7fb5ff to 0x22706b (0x5d4593 bytes) 000304[0] 1734115566.681964 write 0x698ef6 thru 0x699b0b (0xc16 bytes) 000305[1] 1734115566.692898 write 0x2694e3 thru 0x279267 (0xfd85 bytes) 000307[1] 1734115566.714732 read 0x2a1516 thru 0x2aed88 (0xd873 bytes) 000310[0] 1734115566.804983 write 0x3ec15c thru 0x3f5242 (0x90e7 bytes) 000311[0] 1734115566.824035 write 0x1243e3 thru 0x12f46c (0xb08a bytes) 000312[1] 1734115566.830342 read 0x1f2533 thru 0x20107b (0xeb49 bytes) 000314[1] 1734115566.897545 mapread 0x19864f thru 0x1a1249 (0x8bfb bytes) 000315[1] 1734115566.906708 mapwrite 0x970a3d thru 0x97ae77 (0xa43b bytes) 000316[0] 1734115567.009452 mapwrite 0x0c9c9d thru 0x0cf0bb (0x541f bytes) 000317[1] 1734115567.080747 write 0x370a9f thru 0x375bc4 (0x5126 bytes) 000319[1] 1734115567.103180 write 0x47ada6 thru 0x47e0d4 (0x332f bytes) 000320[1] 1734115567.110444 mapwrite 0x5c3743 thru 0x5cdf11 (0xa7cf bytes) 000323[0] 1734115567.140716 read 0x082571 thru 0x0829c7 (0x457 bytes) 000324[1] 1734115567.189468 mapread 0x0077cc thru 0x0153c7 (0xdbfc bytes) 000326[0] 1734115567.236110 trunc from 0x97ae78 to 0x86a8d4 (0x1105a3 bytes) 000328[0] 1734115567.288826 mapread 0x590604 thru 0x5942d6 (0x3cd3 bytes) 000333[1] 1734115567.355927 mapwrite 0x49566d thru 0x4a06ff (0xb093 bytes) 000334[1] 1734115567.442779 mapread 0x5be432 thru 0x5ca904 (0xc4d3 bytes) 000335[1] 1734115567.504137 read 0x39596b thru 0x39842c (0x2ac2 bytes) 000336[0] 1734115567.578678 read 0x5ec3c8 thru 0x5ef3c0 (0x2ff9 bytes) 000337[0] 1734115567.645080 write 0x1a1c26 thru 0x1a774d (0x5b28 bytes) 000338[1] 1734115567.663429 trunc from 0x86a8d5 to 0x8cc3ad (0x61ad9 bytes) 000339[0] 1734115567.709699 mapwrite 0x7181ed thru 0x72082e (0x8642 bytes) 000340[1] 1734115567.806601 write 0x6e265a thru 0x6ec945 (0xa2ec bytes) 000342[1] 1734115567.833566 mapwrite 0x6cd6cb thru 0x6da406 (0xcd3c bytes) 000343[1] 1734115567.926973 write 0x5d9dd1 thru 0x5e73c5 (0xd5f5 bytes) 000344[0] 1734115567.943648 mapread 0x4e51fe thru 0x4e8188 (0x2f8b bytes) 000346[0] 1734115567.974992 mapwrite 0x853352 thru 0x85dab8 (0xa767 bytes) 000348[0] 1734115568.052280 write 0x3c1050 thru 0x3c3e05 (0x2db6 bytes) 000349[1] 1734115568.059150 read 0x761b5b thru 0x767477 (0x591d bytes) 000350[0] 1734115568.073200 mapread 0x6f2e96 thru 0x6f67ce (0x3939 bytes) 000351[0] 1734115568.092336 read 0x57e697 thru 0x582c49 (0x45b3 bytes) 000355[1] 1734115568.156632 trunc from 0x8cc3ae to 0x7fa35e (0xd204f bytes) 000356[1] 1734115568.196035 write 0x2a53f5 thru 0x2ad76c (0x8378 bytes) 000357[1] 1734115568.224690 trunc from 0x7fa35f to 0x034f29 (0x7c5435 bytes) 000358[1] 1734115568.237707 write 0x3b5d62 thru 0x3c0f6d (0xb20c bytes) 000360[0] 1734115568.248999 mapwrite 0x9607ac thru 0x969113 (0x8968 bytes) 000361[0] 1734115568.337328 mapwrite 0x6d3724 thru 0x6dc4b0 (0x8d8d bytes) 000363[1] 1734115568.402319 trunc from 0x969114 to 0x8337ad (0x135966 bytes) 000366[1] 1734115568.433173 read 0x33c7a8 thru 0x345e39 (0x9692 bytes) 000368[1] 1734115568.488181 trunc from 0x8337ae to 0x0bb55d (0x778250 bytes) 000369[0] 1734115568.508187 read 0x0b91b0 thru 0x0ba863 (0x16b4 bytes) 000373[0] 1734115568.541980 mapread 0x0b6028 thru 0x0ba973 (0x494c bytes) 000374[0] 1734115568.547364 read 0x06d80c thru 0x077663 (0x9e58 bytes) 000375[1] 1734115568.553615 read 0x08d103 thru 0x091664 (0x4562 bytes) 000376[0] 1734115568.571607 mapwrite 0x54b2e3 thru 0x54e6cc (0x33ea bytes) 000380[1] 1734115568.684175 read 0x3937af thru 0x396a5a (0x32ac bytes) 000381[1] 1734115568.730041 write 0x0608d8 thru 0x06c2e7 (0xba10 bytes) 000385[0] 1734115568.767913 trunc from 0x54e6cd to 0x64e71d (0x100051 bytes) 000386[1] 1734115568.782541 write 0x6b19b3 thru 0x6b8f4b (0x7599 bytes) 000387[0] 1734115568.793202 write 0x10f171 thru 0x115c0b (0x6a9b bytes) 000389[1] 1734115568.810272 write 0x4571f3 thru 0x45903f (0x1e4d bytes) 000394[0] 1734115568.862975 read 0x4ded5d thru 0x4ec7b2 (0xda56 bytes) 000396[1] 1734115568.921553 mapread 0x57454b thru 0x57cd82 (0x8838 bytes) 000397[1] 1734115568.942439 mapwrite 0x0bd072 thru 0x0c3508 (0x6497 bytes) 000399[0] 1734115569.006842 mapwrite 0x9dd959 thru 0x9e6ece (0x9576 bytes) 000401[0] 1734115569.095227 mapwrite 0x418d44 thru 0x41ab46 (0x1e03 bytes) 000405[0] 1734115569.137797 mapwrite 0x5d0f36 thru 0x5db36a (0xa435 bytes) 000410[1] 1734115569.241054 read 0x83e439 thru 0x843ec6 (0x5a8e bytes) 000411[0] 1734115569.287403 mapwrite 0x748fa5 thru 0x74c17f (0x31db bytes) 000412[1] 1734115569.314855 mapwrite 0x92b726 thru 0x92fb99 (0x4474 bytes) 000413[0] 1734115569.370852 read 0x5c0de4 thru 0x5c431b (0x3538 bytes) 000414[1] 1734115569.385689 mapwrite 0x6c5a8d thru 0x6c63e5 (0x959 bytes) 000421[0] 1734115569.485298 write 0x6b9f48 thru 0x6bd4a5 (0x355e bytes) 000422[0] 1734115569.517131 read 0x07ee0b thru 0x088860 (0x9a56 bytes) 000424[0] 1734115569.608910 trunc from 0x9e6ecf to 0x0ed937 (0x8f9597 bytes) 000425[0] 1734115569.632916 write 0x5c8c04 thru 0x5d214c (0x9549 bytes) 000427[1] 1734115569.653210 mapwrite 0x8d3642 thru 0x8dab0e (0x74cd bytes) 000428[1] 1734115569.744188 write 0x7349c2 thru 0x73624a (0x1889 bytes) 000429[0] 1734115569.751989 read 0x179a19 thru 0x17ffe1 (0x65c9 bytes) 000432[1] 1734115569.806541 mapwrite 0x694174 thru 0x69b99d (0x782a bytes) 000433[0] 1734115569.869818 mapread 0x5f2e25 thru 0x5f5c34 (0x2e10 bytes) 000434[1] 1734115569.913157 read 0x1d1a03 thru 0x1df561 (0xdb5f bytes) 000436[1] 1734115569.940095 mapread 0x86ac16 thru 0x879a53 (0xee3e bytes) 000437[0] 1734115569.963465 mapwrite 0x3895bd thru 0x38c4c9 (0x2f0d bytes) 000443[1] 1734115570.079304 write 0x01ec63 thru 0x0222c2 (0x3660 bytes) 000446[0] 1734115570.097412 mapwrite 0x715d33 thru 0x717d5f (0x202d bytes) 000450[0] 1734115570.196936 read 0x32a6db thru 0x32ccf3 (0x2619 bytes) 000451[0] 1734115570.236549 mapread 0x7e9e78 thru 0x7f8583 (0xe70c bytes) 000455[0] 1734115570.275917 trunc from 0x8dab0f to 0x3d1da7 (0x508d67 bytes) 000458[0] 1734115570.318058 read 0x26c8cd thru 0x27bb41 (0xf275 bytes) 000459[0] 1734115570.361015 write 0x6241d4 thru 0x628a1b (0x4848 bytes) 000460[0] 1734115570.371871 mapread 0x4d7f5d thru 0x4dd108 (0x51ac bytes) 000461[1] 1734115570.396912 write 0x45e47b thru 0x46c15c (0xdce2 bytes) 000464[1] 1734115570.440660 write 0x724ddd thru 0x72ad2b (0x5f4f bytes) 000469[1] 1734115570.480532 mapwrite 0x20bfc8 thru 0x21a7e2 (0xe81b bytes) 000471[1] 1734115570.572677 mapread 0x20e11d thru 0x21c120 (0xe004 bytes) 000475[1] 1734115570.583020 mapread 0x30f6c9 thru 0x31bf04 (0xc83c bytes) 000476[1] 1734115570.637540 mapread 0x02a417 thru 0x03a0a7 (0xfc91 bytes) 000477[0] 1734115570.711550 read 0x4739e5 thru 0x4806af (0xcccb bytes) 000479[1] 1734115570.785384 mapwrite 0x3c19a2 thru 0x3c8a1d (0x707c bytes) 000480[1] 1734115570.845414 mapwrite 0x346f85 thru 0x350bdd (0x9c59 bytes) 000481[0] 1734115570.919882 read 0x1e06a0 thru 0x1eb9a7 (0xb308 bytes) 000484[1] 1734115570.957275 trunc from 0x72ad2c to 0x3483cb (0x3e2960 bytes) 000485[0] 1734115570.987058 mapread 0x28f8c3 thru 0x29f619 (0xfd57 bytes) 000488[0] 1734115571.040296 trunc from 0x3483cc to 0x7a2cbc (0x45a8f1 bytes) 000489[1] 1734115571.062453 mapread 0x15b11a thru 0x16669f (0xb586 bytes) 000490[0] 1734115571.106143 trunc from 0x7a2cbd to 0x6fbec7 (0xa6df5 bytes) 000491[0] 1734115571.126220 mapwrite 0x2c38c3 thru 0x2cd8ef (0xa02d bytes) 000495[1] 1734115571.215122 mapwrite 0x0d2991 thru 0x0dbbc3 (0x9233 bytes) 000496[0] 1734115571.285496 read 0x4a989b thru 0x4ac8ac (0x3012 bytes) 000500[1] 1734115571.308534 write 0x617f25 thru 0x61c34f (0x442b bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5571 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1734115571.641938 write 0x678000 thru 0x6835c6 (0xb5c7 bytes) 000002[0] 1734115571.650846 mapread 0x08b000 thru 0x096082 (0xb083 bytes) truncating to largest ever: 0x79fcb5 000003[1] 1734115571.681767 trunc from 0x6835c7 to 0x79fcb4 (0x11c6ee bytes) 000004[0] 1734115571.692856 mapread 0x784000 thru 0x7916aa (0xd6ab bytes) 000005[1] 1734115571.719076 mapwrite 0x97b000 thru 0x97c049 (0x104a bytes) 000007[0] 1734115571.805659 trunc from 0x97c04a to 0x33e1d5 (0x63de74 bytes) 000008[0] 1734115571.823767 trunc from 0x33e1d6 to 0x6a048e (0x3622b9 bytes) 000011[0] 1734115571.832200 trunc from 0x6a048f to 0x212c67 (0x48d827 bytes) 000017[1] 1734115571.864898 trunc from 0x212c68 to 0x75e8ff (0x54bc98 bytes) 000019[1] 1734115571.893685 mapwrite 0x04e000 thru 0x054f2b (0x6f2c bytes) 000020[1] 1734115571.945376 mapwrite 0x10f000 thru 0x1187ff (0x9800 bytes) 000022[1] 1734115571.993748 mapread 0x3a2000 thru 0x3a3ea1 (0x1ea2 bytes) 000023[0] 1734115571.999771 mapwrite 0x601000 thru 0x60b0c8 (0xa0c9 bytes) 000025[1] 1734115572.063768 mapwrite 0x1e8000 thru 0x1f0728 (0x8729 bytes) 000030[1] 1734115572.103822 read 0x42a000 thru 0x42ee41 (0x4e42 bytes) 000032[1] 1734115572.108763 mapwrite 0x194000 thru 0x19ab34 (0x6b35 bytes) truncating to largest ever: 0x870598 000035[1] 1734115572.147541 trunc from 0x75e900 to 0x870597 (0x111c98 bytes) 000036[1] 1734115572.160956 trunc from 0x870598 to 0x62fa20 (0x240b77 bytes) 000037[0] 1734115572.175745 write 0x562000 thru 0x56ec6c (0xcc6d bytes) 000040[1] 1734115572.196821 trunc from 0x62fa21 to 0x4c0b7b (0x16eea5 bytes) 000041[1] 1734115572.210489 write 0x201000 thru 0x208ba5 (0x7ba6 bytes) 000043[0] 1734115572.214712 trunc from 0x4c0b7c to 0x328c1f (0x197f5c bytes) 000045[1] 1734115572.239026 mapread 0x09b000 thru 0x09cbf0 (0x1bf1 bytes) 000046[0] 1734115572.252034 write 0x390000 thru 0x3941ae (0x41af bytes) 000050[0] 1734115572.268396 mapwrite 0x67f000 thru 0x68282c (0x382d bytes) truncating to largest ever: 0x8d938d 000052[0] 1734115572.332536 trunc from 0x68282d to 0x8d938c (0x256b60 bytes) 000053[1] 1734115572.339781 mapwrite 0x04f000 thru 0x0576f1 (0x86f2 bytes) 000054[0] 1734115572.395867 mapread 0x571000 thru 0x57a3f7 (0x93f8 bytes) 000056[1] 1734115572.419236 write 0x3dd000 thru 0x3ddbd1 (0xbd2 bytes) 000061[0] 1734115572.449947 trunc from 0x8d938d to 0x29192b (0x647a61 bytes) 000065[0] 1734115572.467805 read 0x1f6000 thru 0x203e7b (0xde7c bytes) 000066[0] 1734115572.471804 mapwrite 0x4ca000 thru 0x4d6209 (0xc20a bytes) 000067[1] 1734115572.538396 mapread 0x17c000 thru 0x189987 (0xd988 bytes) 000068[1] 1734115572.564318 read 0x3d5000 thru 0x3dca00 (0x7a01 bytes) 000069[1] 1734115572.599422 mapwrite 0x6fd000 thru 0x705461 (0x8462 bytes) 000070[0] 1734115572.669136 read 0x2db000 thru 0x2ddc60 (0x2c61 bytes) 000071[1] 1734115572.728529 mapread 0x3d7000 thru 0x3db8b3 (0x48b4 bytes) 000072[1] 1734115572.746814 read 0x5a5000 thru 0x5a5c7e (0xc7f bytes) 000074[1] 1734115572.798278 read 0x3a3000 thru 0x3aedbd (0xbdbe bytes) 000075[0] 1734115572.798634 trunc from 0x705462 to 0x0c96b9 (0x63bda8 bytes) 000078[0] 1734115572.823238 trunc from 0x0c96ba to 0x560e23 (0x49776a bytes) 000079[1] 1734115572.837060 write 0x432000 thru 0x43d163 (0xb164 bytes) 000080[0] 1734115572.870364 write 0x317000 thru 0x31b40e (0x440f bytes) 000082[0] 1734115572.899782 mapwrite 0x9ef000 thru 0x9f1ccb (0x2ccc bytes) 000084[1] 1734115572.986775 read 0x559000 thru 0x55e2e6 (0x52e7 bytes) 000087[0] 1734115573.065054 trunc from 0x9f1ccc to 0x448737 (0x5a9594 bytes) 000089[1] 1734115573.103210 mapwrite 0x0fa000 thru 0x0ffabc (0x5abd bytes) 000090[0] 1734115573.158274 read 0x1b3000 thru 0x1b6609 (0x360a bytes) 000092[0] 1734115573.195959 mapread 0x2d8000 thru 0x2e4a2b (0xca2c bytes) 000095[0] 1734115573.196571 mapread 0x05a000 thru 0x062fee (0x8fef bytes) 000096[1] 1734115573.196933 read 0x433000 thru 0x435178 (0x2179 bytes) 000102[1] 1734115573.227469 mapread 0x2d3000 thru 0x2d5b24 (0x2b25 bytes) 000103[1] 1734115573.227950 write 0x5cb000 thru 0x5ce326 (0x3327 bytes) 000104[0] 1734115573.246059 write 0x431000 thru 0x431f9d (0xf9e bytes) 000105[1] 1734115573.268670 mapread 0x0cf000 thru 0x0db93a (0xc93b bytes) 000106[1] 1734115573.320074 mapwrite 0x7de000 thru 0x7ebf70 (0xdf71 bytes) 000107[1] 1734115573.386629 trunc from 0x7ebf71 to 0x7ed3f7 (0x1487 bytes) 000109[0] 1734115573.409648 mapwrite 0x282000 thru 0x2882f1 (0x62f2 bytes) 000110[0] 1734115573.485157 mapread 0x17f000 thru 0x185a9e (0x6a9f bytes) 000112[0] 1734115573.514383 read 0x61f000 thru 0x62abab (0xbbac bytes) 000113[1] 1734115573.521144 mapwrite 0x515000 thru 0x520b05 (0xbb06 bytes) 000114[0] 1734115573.606858 read 0x3a9000 thru 0x3ab709 (0x270a bytes) 000115[0] 1734115573.657011 trunc from 0x7ed3f8 to 0x3f52af (0x3f8148 bytes) 000117[0] 1734115573.687408 write 0x1d1000 thru 0x1d7068 (0x6069 bytes) 000118[1] 1734115573.690558 read 0x30a000 thru 0x30ab9f (0xba0 bytes) 000120[0] 1734115573.732897 write 0x689000 thru 0x68d0b6 (0x40b7 bytes) 000121[1] 1734115573.747478 trunc from 0x68d0b7 to 0x736697 (0xa95e1 bytes) 000123[1] 1734115573.771797 mapwrite 0x56d000 thru 0x57723f (0xa240 bytes) 000124[1] 1734115573.836865 mapread 0x4ce000 thru 0x4dd39e (0xf39f bytes) 000127[1] 1734115573.842811 write 0x153000 thru 0x15fd00 (0xcd01 bytes) 000130[1] 1734115573.858914 mapwrite 0x400000 thru 0x40bdc6 (0xbdc7 bytes) 000131[1] 1734115573.892691 read 0x23d000 thru 0x243a40 (0x6a41 bytes) 000132[1] 1734115573.926525 mapwrite 0x54e000 thru 0x554191 (0x6192 bytes) 000133[1] 1734115573.953553 mapwrite 0x2e6000 thru 0x2efeec (0x9eed bytes) 000136[0] 1734115573.993087 trunc from 0x736698 to 0x372f90 (0x3c3707 bytes) 000140[1] 1734115574.022311 mapwrite 0x139000 thru 0x13bfc3 (0x2fc4 bytes) 000141[0] 1734115574.055371 write 0x462000 thru 0x4677b7 (0x57b8 bytes) 000142[1] 1734115574.059830 trunc from 0x4677b8 to 0x65ca19 (0x1f5262 bytes) 000145[0] 1734115574.079340 read 0x37f000 thru 0x389fac (0xafad bytes) 000146[0] 1734115574.120157 write 0x9f0000 thru 0x9f9cbf (0x9cc0 bytes) 000147[1] 1734115574.138255 read 0x5f6000 thru 0x5f9e4e (0x3e4f bytes) 000151[0] 1734115574.203145 read 0x6c5000 thru 0x6c752f (0x2530 bytes) 000153[1] 1734115574.256278 trunc from 0x9f9cc0 to 0x3f5bad (0x604112 bytes) 000154[0] 1734115574.276209 write 0x3f8000 thru 0x3fe38b (0x638c bytes) 000156[0] 1734115574.289869 read 0x279000 thru 0x28270e (0x970f bytes) 000160[0] 1734115574.328713 mapread 0x266000 thru 0x267201 (0x1202 bytes) 000161[1] 1734115574.329065 read 0x37d000 thru 0x38abcc (0xdbcd bytes) 000162[0] 1734115574.369252 read 0x3f2000 thru 0x3fde1f (0xbe20 bytes) 000163[1] 1734115574.396743 write 0x7b1000 thru 0x7b619e (0x519f bytes) 000164[1] 1734115574.406327 mapwrite 0x7db000 thru 0x7e3f47 (0x8f48 bytes) 000167[0] 1734115574.495210 mapwrite 0x1f2000 thru 0x1fc8ea (0xa8eb bytes) 000169[0] 1734115574.594210 mapread 0x4d7000 thru 0x4e482f (0xd830 bytes) 000174[1] 1734115574.609983 write 0x22e000 thru 0x23cbab (0xebac bytes) 000175[0] 1734115574.621574 write 0x1bd000 thru 0x1c5058 (0x8059 bytes) 000176[1] 1734115574.635262 write 0x43b000 thru 0x445d38 (0xad39 bytes) 000177[1] 1734115574.643114 read 0x2fd000 thru 0x306e45 (0x9e46 bytes) 000178[1] 1734115574.701713 write 0x4e3000 thru 0x4ee499 (0xb49a bytes) 000181[1] 1734115574.719308 mapwrite 0x351000 thru 0x35d3a6 (0xc3a7 bytes) 000183[0] 1734115574.759981 mapread 0x1ce000 thru 0x1dafa1 (0xcfa2 bytes) 000184[1] 1734115574.800799 mapwrite 0x4a3000 thru 0x4a8e82 (0x5e83 bytes) 000185[1] 1734115574.850184 mapwrite 0x504000 thru 0x50ad76 (0x6d77 bytes) 000186[0] 1734115574.888400 write 0x8a5000 thru 0x8aad06 (0x5d07 bytes) 000189[0] 1734115574.906564 mapwrite 0x73b000 thru 0x73eca9 (0x3caa bytes) truncating to largest ever: 0x91f4ac 000195[0] 1734115574.954941 trunc from 0x8aad07 to 0x91f4ab (0x747a5 bytes) 000196[1] 1734115574.961204 trunc from 0x91f4ac to 0x828a3c (0xf6a6f bytes) 000198[0] 1734115574.982165 read 0x08d000 thru 0x097141 (0xa142 bytes) 000199[0] 1734115575.025461 write 0x815000 thru 0x81ff1d (0xaf1e bytes) 000201[1] 1734115575.057840 trunc from 0x828a3d to 0x52b41c (0x2fd620 bytes) 000203[1] 1734115575.072133 mapread 0x135000 thru 0x142a7a (0xda7b bytes) 000205[0] 1734115575.107246 read 0x054000 thru 0x05ca2a (0x8a2b bytes) 000208[0] 1734115575.143982 trunc from 0x52b41d to 0x2b4c52 (0x2767ca bytes) 000210[1] 1734115575.165480 trunc from 0x2b4c53 to 0x27c8ab (0x383a7 bytes) truncating to largest ever: 0x99bbd9 000211[0] 1734115575.181133 trunc from 0x27c8ac to 0x99bbd8 (0x71f32d bytes) 000214[0] 1734115575.207954 read 0x73b000 thru 0x73f732 (0x4733 bytes) 000217[0] 1734115575.271375 trunc from 0x99bbd9 to 0x15a24f (0x841989 bytes) 000218[1] 1734115575.291268 write 0x345000 thru 0x346791 (0x1792 bytes) 000219[1] 1734115575.295975 write 0x098000 thru 0x0a5462 (0xd463 bytes) 000220[1] 1734115575.297828 read 0x192000 thru 0x19ce58 (0xae59 bytes) 000221[1] 1734115575.318996 write 0x2e5000 thru 0x2ead19 (0x5d1a bytes) 000222[0] 1734115575.319586 read 0x0a6000 thru 0x0b1b89 (0xbb8a bytes) 000224[1] 1734115575.365421 mapwrite 0x53e000 thru 0x53e636 (0x637 bytes) 000227[1] 1734115575.444056 read 0x02d000 thru 0x03c4e5 (0xf4e6 bytes) 000228[0] 1734115575.462837 mapwrite 0x298000 thru 0x2a588c (0xd88d bytes) 000230[1] 1734115575.545908 write 0x804000 thru 0x8095af (0x55b0 bytes) 000232[0] 1734115575.557623 write 0x233000 thru 0x23a669 (0x766a bytes) 000233[0] 1734115575.590169 write 0x7d3000 thru 0x7e20d6 (0xf0d7 bytes) 000237[1] 1734115575.603405 write 0x443000 thru 0x451acc (0xeacd bytes) 000242[1] 1734115575.644136 write 0x52f000 thru 0x53d49c (0xe49d bytes) 000243[1] 1734115575.653068 mapread 0x2c5000 thru 0x2d0cbf (0xbcc0 bytes) 000245[0] 1734115575.674993 mapread 0x001000 thru 0x007423 (0x6424 bytes) 000246[0] 1734115575.702980 trunc from 0x8095b0 to 0x0a986c (0x75fd43 bytes) 000247[1] 1734115575.724743 mapread 0x07a000 thru 0x089600 (0xf601 bytes) 000254[1] 1734115575.772664 mapread 0x022000 thru 0x02b288 (0x9289 bytes) 000259[0] 1734115575.794159 read 0x071000 thru 0x07db81 (0xcb82 bytes) 000261[0] 1734115575.805420 mapread 0x071000 thru 0x0760b0 (0x50b1 bytes) 000262[1] 1734115575.805808 read 0x003000 thru 0x0097ff (0x6800 bytes) 000264[0] 1734115575.814237 trunc from 0x0a986d to 0x944bd8 (0x89b36c bytes) 000265[1] 1734115575.835361 mapwrite 0x2bd000 thru 0x2cbcb5 (0xecb6 bytes) 000268[1] 1734115575.937488 mapread 0x696000 thru 0x69b564 (0x5565 bytes) 000271[0] 1734115575.949782 read 0x1bb000 thru 0x1c7c04 (0xcc05 bytes) 000272[1] 1734115576.015492 write 0x045000 thru 0x04e935 (0x9936 bytes) 000275[1] 1734115576.041954 trunc from 0x944bd9 to 0x51b681 (0x429557 bytes) 000276[0] 1734115576.052206 mapread 0x39d000 thru 0x39e6ea (0x16eb bytes) 000280[0] 1734115576.071060 write 0x5d2000 thru 0x5d4adb (0x2adc bytes) 000281[1] 1734115576.075483 write 0x451000 thru 0x45f108 (0xe109 bytes) 000282[1] 1734115576.090085 read 0x582000 thru 0x58588c (0x388d bytes) 000284[1] 1734115576.146876 write 0x88f000 thru 0x88fbff (0xc00 bytes) 000285[0] 1734115576.153016 mapwrite 0x101000 thru 0x1073d1 (0x63d2 bytes) 000287[0] 1734115576.210091 trunc from 0x88fc00 to 0x98f815 (0xffc16 bytes) 000289[0] 1734115576.229341 write 0x0b3000 thru 0x0b6ac1 (0x3ac2 bytes) 000290[1] 1734115576.238080 write 0x7c3000 thru 0x7ce7ed (0xb7ee bytes) 000291[1] 1734115576.264089 mapwrite 0x7d7000 thru 0x7e2b24 (0xbb25 bytes) 000295[1] 1734115576.323274 write 0x510000 thru 0x5196c1 (0x96c2 bytes) 000297[1] 1734115576.332066 write 0x2f9000 thru 0x3085b8 (0xf5b9 bytes) 000298[0] 1734115576.342087 mapread 0x089000 thru 0x089b5f (0xb60 bytes) 000300[1] 1734115576.364152 mapwrite 0x411000 thru 0x41ac11 (0x9c12 bytes) 000301[1] 1734115576.434040 mapwrite 0x1e0000 thru 0x1eb85f (0xb860 bytes) 000302[1] 1734115576.497822 mapread 0x4b5000 thru 0x4b5baf (0xbb0 bytes) 000303[0] 1734115576.508113 mapwrite 0x268000 thru 0x268de4 (0xde5 bytes) 000304[0] 1734115576.546706 write 0x661000 thru 0x66c79d (0xb79e bytes) 000305[0] 1734115576.556412 read 0x001000 thru 0x00428c (0x328d bytes) 000306[0] 1734115576.612450 trunc from 0x98f816 to 0x836ca4 (0x158b71 bytes) 000307[0] 1734115576.622334 mapwrite 0x4e9000 thru 0x4f1752 (0x8753 bytes) 000311[1] 1734115576.681804 trunc from 0x836ca5 to 0x8818ec (0x4ac48 bytes) 000313[0] 1734115576.712816 mapwrite 0x362000 thru 0x3653fc (0x33fd bytes) 000315[1] 1734115576.766898 read 0x723000 thru 0x728b76 (0x5b77 bytes) 000316[1] 1734115576.824957 read 0x396000 thru 0x39b4e0 (0x54e1 bytes) 000318[1] 1734115576.825439 write 0x869000 thru 0x86d125 (0x4126 bytes) 000319[1] 1734115576.890601 write 0x1fa000 thru 0x204bd0 (0xabd1 bytes) 000320[0] 1734115576.891327 mapread 0x4c2000 thru 0x4c8225 (0x6226 bytes) 000322[0] 1734115576.920965 mapread 0x873000 thru 0x87c87c (0x987d bytes) 000323[0] 1734115576.938994 mapread 0x223000 thru 0x22b57d (0x857e bytes) 000324[1] 1734115576.954068 read 0x0c3000 thru 0x0cd026 (0xa027 bytes) 000326[0] 1734115576.985295 mapwrite 0x1e9000 thru 0x1ec49e (0x349f bytes) 000328[0] 1734115577.042024 mapwrite 0x53b000 thru 0x5429ee (0x79ef bytes) 000329[1] 1734115577.087604 mapread 0x29a000 thru 0x2a303d (0x903e bytes) 000333[1] 1734115577.115461 trunc from 0x8818ed to 0x195977 (0x6ebf75 bytes) 000336[1] 1734115577.145775 mapread 0x0a7000 thru 0x0aa556 (0x3557 bytes) 000339[1] 1734115577.163377 mapread 0x163000 thru 0x1641f0 (0x11f1 bytes) 000343[1] 1734115577.175410 read 0x05f000 thru 0x065312 (0x6313 bytes) 000347[0] 1734115577.208236 read 0x11f000 thru 0x123de2 (0x4de3 bytes) 000349[1] 1734115577.233170 mapread 0x18e000 thru 0x18e14b (0x14c bytes) 000350[1] 1734115577.235855 trunc from 0x195978 to 0x08a8f6 (0x10b081 bytes) 000352[1] 1734115577.256039 trunc from 0x08a8f7 to 0x153e27 (0xc9531 bytes) 000358[0] 1734115577.287043 read 0x0e1000 thru 0x0e888a (0x788b bytes) 000359[1] 1734115577.305590 read 0x137000 thru 0x137387 (0x388 bytes) 000360[0] 1734115577.322449 read 0x0b8000 thru 0x0b98a4 (0x18a5 bytes) 000364[0] 1734115577.323004 mapread 0x0e8000 thru 0x0f03b7 (0x83b8 bytes) 000366[1] 1734115577.323633 mapwrite 0x540000 thru 0x54e425 (0xe426 bytes) 000367[0] 1734115577.403974 mapread 0x495000 thru 0x4a34fb (0xe4fc bytes) 000370[1] 1734115577.448403 read 0x2c5000 thru 0x2d2f00 (0xdf01 bytes) 000373[1] 1734115577.477357 write 0x442000 thru 0x4472ed (0x52ee bytes) 000374[1] 1734115577.493775 trunc from 0x54e426 to 0x910485 (0x3c2060 bytes) 000376[0] 1734115577.510721 write 0x86c000 thru 0x86fa1c (0x3a1d bytes) 000377[0] 1734115577.539245 mapread 0x35c000 thru 0x363deb (0x7dec bytes) 000380[0] 1734115577.583132 write 0x0e3000 thru 0x0e363c (0x63d bytes) 000381[1] 1734115577.586149 mapread 0x8a5000 thru 0x8a776c (0x276d bytes) 000382[0] 1734115577.631885 mapread 0x389000 thru 0x394a85 (0xba86 bytes) 000383[1] 1734115577.654036 mapwrite 0x218000 thru 0x21c3ea (0x43eb bytes) 000384[1] 1734115577.707934 mapread 0x903000 thru 0x90f691 (0xc692 bytes) 000385[0] 1734115577.753155 trunc from 0x910486 to 0x2203c7 (0x6f00be bytes) 000388[1] 1734115577.784225 trunc from 0x2203c8 to 0x5fa13a (0x3d9d73 bytes) 000389[0] 1734115577.801879 write 0x9fc000 thru 0x9ff3b7 (0x33b8 bytes) 000390[1] 1734115577.811266 read 0x73b000 thru 0x746572 (0xb573 bytes) 000391[0] 1734115577.880304 trunc from 0x9ff3b8 to 0x062291 (0x99d126 bytes) 000400[1] 1734115577.921144 read 0x048000 thru 0x04c96e (0x496f bytes) 000412[1] 1734115577.937464 mapread 0x001000 thru 0x00749b (0x649c bytes) 000413[0] 1734115577.937984 read 0x01d000 thru 0x028646 (0xb647 bytes) 000421[1] 1734115577.946649 mapwrite 0x4da000 thru 0x4e8a41 (0xea42 bytes) 000423[1] 1734115578.030832 mapread 0x247000 thru 0x2545d4 (0xd5d5 bytes) 000426[1] 1734115578.044827 read 0x118000 thru 0x11e4f6 (0x64f7 bytes) 000427[0] 1734115578.045176 read 0x0cd000 thru 0x0d3243 (0x6244 bytes) 000429[1] 1734115578.095047 mapread 0x340000 thru 0x34e79b (0xe79c bytes) 000433[0] 1734115578.139591 mapwrite 0x1df000 thru 0x1e2361 (0x3362 bytes) 000439[0] 1734115578.229201 read 0x132000 thru 0x134d12 (0x2d13 bytes) 000440[0] 1734115578.263998 write 0x62d000 thru 0x638202 (0xb203 bytes) 000441[1] 1734115578.276457 write 0x7c1000 thru 0x7d03d5 (0xf3d6 bytes) 000442[0] 1734115578.291832 trunc from 0x7d03d6 to 0x5f09f1 (0x1df9e4 bytes) 000445[1] 1734115578.312648 write 0x592000 thru 0x5a00d6 (0xe0d7 bytes) 000447[1] 1734115578.338782 write 0x995000 thru 0x9a03db (0xb3dc bytes) 000448[1] 1734115578.340183 read 0x908000 thru 0x90a8a4 (0x28a5 bytes) 000453[1] 1734115578.430417 read 0x246000 thru 0x24cfc4 (0x6fc5 bytes) 000454[1] 1734115578.438191 mapwrite 0x7b0000 thru 0x7b9a0c (0x9a0d bytes) 000455[0] 1734115578.489387 mapread 0x292000 thru 0x29958b (0x758c bytes) 000459[0] 1734115578.570865 mapread 0x78a000 thru 0x78ddc3 (0x3dc4 bytes) 000460[0] 1734115578.571358 mapwrite 0x331000 thru 0x33d684 (0xc685 bytes) 000462[0] 1734115578.675408 mapread 0x532000 thru 0x53a85d (0x885e bytes) 000463[1] 1734115578.705887 write 0x490000 thru 0x49ab3a (0xab3b bytes) 000464[0] 1734115578.757347 mapread 0x12d000 thru 0x139893 (0xc894 bytes) 000466[0] 1734115578.814968 write 0x22a000 thru 0x22f52f (0x5530 bytes) 000468[1] 1734115578.838754 trunc from 0x9a03dc to 0x3e8904 (0x5b7ad7 bytes) 000471[0] 1734115578.867829 mapread 0x0d7000 thru 0x0e44a5 (0xd4a6 bytes) 000472[0] 1734115578.905026 trunc from 0x3e8905 to 0x2e4b3c (0x103dc8 bytes) 000476[0] 1734115578.942000 read 0x161000 thru 0x16294b (0x194c bytes) 000478[0] 1734115578.972807 mapread 0x0d5000 thru 0x0dea89 (0x9a8a bytes) 000480[0] 1734115578.987941 mapwrite 0x2e2000 thru 0x2ecd08 (0xad09 bytes) 000481[0] 1734115579.039258 mapread 0x030000 thru 0x036454 (0x6455 bytes) 000482[0] 1734115579.039689 mapwrite 0x855000 thru 0x860d3a (0xbd3b bytes) 000483[0] 1734115579.115097 write 0x63d000 thru 0x64254b (0x554c bytes) 000484[1] 1734115579.115878 mapwrite 0x805000 thru 0x808644 (0x3645 bytes) 000485[1] 1734115579.188699 mapread 0x665000 thru 0x669d4c (0x4d4d bytes) 000488[0] 1734115579.197568 write 0x637000 thru 0x643e3c (0xce3d bytes) 000489[0] 1734115579.221697 mapread 0x567000 thru 0x56ab83 (0x3b84 bytes) 000490[0] 1734115579.232174 trunc from 0x860d3b to 0x888ad1 (0x27d97 bytes) 000493[1] 1734115579.254750 write 00000000 thru 0x00fa73 (0xfa74 bytes) 000497[1] 1734115579.282056 write 0x1f5000 thru 0x1fd188 (0x8189 bytes) 000498[0] 1734115579.291845 read 0x4e9000 thru 0x4f061c (0x761d bytes) 000499[0] 1734115579.346664 trunc from 0x888ad2 to 0x8d5078 (0x4c5a7 bytes) All operations completed A-OK! PASS 16a (21s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 13:46:20 (1734115580) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.217316 s, 38.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.203537 s, 41.2 MB/s Chance of close/open is 1 in 50 Seed set to 5581 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000009[0] 1734115582.085350 mapwrite 0x9c4071 thru 0x9c45a3 (0x533 bytes) 000010[0] 1734115582.154301 read 0x973239 thru 0x973dc3 (0xb8b bytes) 000011[1] 1734115582.158233 mapwrite 0x7ac742 thru 0x7add10 (0x15cf bytes) 000015[1] 1734115582.219783 read 0x30b5a9 thru 0x30c881 (0x12d9 bytes) 000016[1] 1734115582.226451 write 0x58cb98 thru 0x58eaa5 (0x1f0e bytes) 000017[1] 1734115582.240381 write 0x014a3a thru 0x015f98 (0x155f bytes) truncating to largest ever: 0x8ff4b9 000018[1] 1734115582.249812 trunc from 0x9c45a4 to 0x8ff4b8 (0xc50eb bytes) 000020[0] 1734115582.266232 mapwrite 0x8b82fd thru 0x8b937b (0x107f bytes) 000023[0] 1734115582.326940 mapwrite 0x5d8309 thru 0x5d9b65 (0x185d bytes) 000027[1] 1734115582.370168 mapwrite 0x878d44 thru 0x8791bb (0x478 bytes) 000028[1] 1734115582.409749 read 0x261fea thru 0x262f99 (0xfb0 bytes) 000029[0] 1734115582.448394 mapread 0x852523 thru 0x85335b (0xe39 bytes) 000030[0] 1734115582.463734 write 0x034a53 thru 0x034c51 (0x1ff bytes) 000032[1] 1734115582.476053 write 0x431c9c thru 0x431d5b (0xc0 bytes) 000033[1] 1734115582.517845 write 0x99f5b7 thru 0x9a13d4 (0x1e1e bytes) 000034[1] 1734115582.531990 write 0x244faa thru 0x246d50 (0x1da7 bytes) 000039[1] 1734115582.568037 trunc from 0x9a13d5 to 0x0ee6ee (0x8b2ce6 bytes) 000042[1] 1734115582.602030 trunc from 0x0ee6ef to 0x824174 (0x735a86 bytes) 000044[1] 1734115582.612842 trunc from 0x824175 to 0x6884c4 (0x19bcb0 bytes) 000045[0] 1734115582.620605 trunc from 0x6884c5 to 0x1ef9cc (0x498af8 bytes) 000046[0] 1734115582.642128 trunc from 0x1ef9cd to 0x368f42 (0x179576 bytes) 000048[1] 1734115582.650794 mapwrite 0x6fb37a thru 0x6fc395 (0x101c bytes) truncating to largest ever: 0x9d182d 000051[0] 1734115582.713952 trunc from 0x6fc396 to 0x9d182c (0x2d5497 bytes) 000052[0] 1734115582.734329 read 0x7a8eb8 thru 0x7a9dea (0xf33 bytes) 000053[0] 1734115582.743065 read 0x6cbb1d thru 0x6cc2ee (0x7d2 bytes) 000054[0] 1734115582.787014 trunc from 0x9d182d to 0x68b72a (0x346102 bytes) 000055[0] 1734115582.800482 mapwrite 0x45933c thru 0x459cb3 (0x978 bytes) 000056[1] 1734115582.821090 read 0x5fe36b thru 0x5ff466 (0x10fc bytes) 000060[0] 1734115582.847486 write 0x3c9cd4 thru 0x3cae4d (0x117a bytes) 000067[1] 1734115582.874223 mapwrite 0x1e9cdb thru 0x1eacbe (0xfe4 bytes) 000069[0] 1734115582.945469 mapwrite 0x2bd76f thru 0x2bf3b8 (0x1c4a bytes) 000072[0] 1734115583.001734 write 0x16f9f0 thru 0x17148d (0x1a9e bytes) 000074[1] 1734115583.022211 read 0x3a2d74 thru 0x3a30c3 (0x350 bytes) 000076[1] 1734115583.058741 write 0x979551 thru 0x97af54 (0x1a04 bytes) 000080[0] 1734115583.080658 read 0x592452 thru 0x593387 (0xf36 bytes) 000081[1] 1734115583.087767 mapread 0x687119 thru 0x687524 (0x40c bytes) 000082[1] 1734115583.088252 read 0x975764 thru 0x977297 (0x1b34 bytes) 000083[1] 1734115583.121209 read 0x66bbfd thru 0x66c2ae (0x6b2 bytes) 000084[0] 1734115583.129258 mapread 0x3c5dbd thru 0x3c694f (0xb93 bytes) 000087[0] 1734115583.172463 trunc from 0x97af55 to 0x8a3843 (0xd7711 bytes) 000089[1] 1734115583.213343 mapread 0x48b428 thru 0x48b7e5 (0x3be bytes) 000090[0] 1734115583.224648 write 0x689a1f thru 0x68aeed (0x14cf bytes) 000091[1] 1734115583.260716 read 0x755a08 thru 0x755e5d (0x456 bytes) 000092[0] 1734115583.282640 mapread 0x131df8 thru 0x131fee (0x1f7 bytes) 000093[0] 1734115583.293056 mapread 0x23a9b1 thru 0x23b3d4 (0xa24 bytes) 000094[1] 1734115583.297128 mapread 0x2d4225 thru 0x2d4b2f (0x90b bytes) 000095[0] 1734115583.328893 trunc from 0x8a3844 to 0x976468 (0xd2c25 bytes) 000097[1] 1734115583.347219 trunc from 0x976469 to 0x6dfa6e (0x2969fa bytes) 000098[1] 1734115583.366861 trunc from 0x6dfa6f to 0x444942 (0x29b12c bytes) 000099[1] 1734115583.380200 mapread 0x0be5ae thru 0x0bf79a (0x11ed bytes) 000100[1] 1734115583.395680 trunc from 0x444943 to 0x0dc0cc (0x368876 bytes) 000101[1] 1734115583.410589 trunc from 0x0dc0cd to 0x8a5984 (0x7c98b8 bytes) 000102[0] 1734115583.418131 mapread 0x30307b thru 0x3030c9 (0x4f bytes) 000104[0] 1734115583.437122 mapread 0x4a350b thru 0x4a4ce6 (0x17dc bytes) 000108[0] 1734115583.448046 mapwrite 0x761570 thru 0x76197f (0x410 bytes) 000110[1] 1734115583.487867 mapwrite 0x924f27 thru 0x9263c4 (0x149e bytes) 000114[1] 1734115583.568934 mapread 0x38e220 thru 0x38e3a7 (0x188 bytes) 000115[0] 1734115583.572365 trunc from 0x9263c5 to 0x32e8a1 (0x5f7b23 bytes) 000117[0] 1734115583.599116 write 0x846d22 thru 0x84841d (0x16fc bytes) 000118[1] 1734115583.600879 mapread 0x24311d thru 0x243393 (0x277 bytes) 000119[1] 1734115583.612292 mapwrite 0x8a1b91 thru 0x8a2d07 (0x1177 bytes) 000120[1] 1734115583.675014 trunc from 0x8a2d08 to 0x465e38 (0x43cecf bytes) 000122[1] 1734115583.683847 mapread 0x2261b7 thru 0x226d94 (0xbde bytes) 000124[0] 1734115583.698409 mapread 0x34000c thru 0x3414ea (0x14df bytes) 000125[0] 1734115583.710875 read 0x29a830 thru 0x29b518 (0xce9 bytes) 000126[1] 1734115583.750108 mapread 0x20b5b8 thru 0x20bcd7 (0x720 bytes) 000132[1] 1734115583.780934 mapread 0x03b9ae thru 0x03cd73 (0x13c6 bytes) 000133[1] 1734115583.789820 mapwrite 0x8915f1 thru 0x8924ae (0xebe bytes) 000134[0] 1734115583.839388 write 0x92cdd2 thru 0x92e92d (0x1b5c bytes) 000136[1] 1734115583.858072 read 0x7387f2 thru 0x73a1f3 (0x1a02 bytes) 000139[1] 1734115583.880409 mapwrite 0x168997 thru 0x1690ba (0x724 bytes) 000142[0] 1734115583.963559 mapread 0x5e4454 thru 0x5e54cd (0x107a bytes) 000143[0] 1734115584.010030 trunc from 0x92e92e to 0x32138c (0x60d5a1 bytes) 000146[1] 1734115584.053462 mapwrite 0x3c43f0 thru 0x3c60e7 (0x1cf8 bytes) 000148[1] 1734115584.108203 trunc from 0x3c60e8 to 0x3736bc (0x52a2b bytes) 000149[0] 1734115584.126137 mapwrite 0x176052 thru 0x176821 (0x7d0 bytes) 000150[0] 1734115584.173286 read 0x1a2111 thru 0x1a3092 (0xf82 bytes) 000152[1] 1734115584.215648 trunc from 0x3736bd to 0x6cb0e1 (0x357a25 bytes) 000153[1] 1734115584.236377 mapwrite 0x42918a thru 0x42a620 (0x1497 bytes) 000156[0] 1734115584.306146 read 0x364de6 thru 0x364ffa (0x215 bytes) 000157[0] 1734115584.337078 mapwrite 0x078fb5 thru 0x079baa (0xbf6 bytes) 000160[1] 1734115584.404298 mapread 0x0f5260 thru 0x0f549e (0x23f bytes) 000164[1] 1734115584.424180 write 0x795ae9 thru 0x79797a (0x1e92 bytes) 000165[1] 1734115584.430898 trunc from 0x79797b to 0x776a21 (0x20f59 bytes) 000168[0] 1734115584.441922 mapwrite 0x088fa8 thru 0x0897a7 (0x800 bytes) 000172[1] 1734115584.497650 mapwrite 0x9e8ef6 thru 0x9ea031 (0x113c bytes) 000174[0] 1734115584.569030 mapwrite 0x6a260b thru 0x6a3134 (0xb2a bytes) 000175[0] 1734115584.647974 write 0x07277b thru 0x074026 (0x18ac bytes) 000176[0] 1734115584.682914 mapread 0x0e0df5 thru 0x0e1e0b (0x1017 bytes) 000178[1] 1734115584.703560 write 0x3e7cd2 thru 0x3e8eeb (0x121a bytes) 000180[0] 1734115584.732377 write 0x90a0eb thru 0x90aee2 (0xdf8 bytes) 000181[1] 1734115584.750699 write 0x36a621 thru 0x36b4ec (0xecc bytes) 000183[0] 1734115584.778142 mapwrite 0x79657d thru 0x797d9d (0x1821 bytes) 000185[0] 1734115584.819823 mapwrite 0x41d635 thru 0x41eaa1 (0x146d bytes) 000188[0] 1734115584.865142 write 0x3b4eaf thru 0x3b5be7 (0xd39 bytes) 000190[1] 1734115584.891235 write 0x275da2 thru 0x276518 (0x777 bytes) 000196[1] 1734115584.946064 trunc from 0x9ea032 to 0x9c806e (0x21fc3 bytes) 000197[0] 1734115584.973070 write 0x1714da thru 0x1720b6 (0xbdd bytes) 000201[0] 1734115585.004941 read 0x1f0dad thru 0x1f214b (0x139f bytes) 000202[0] 1734115585.038735 mapread 0x14f4bb thru 0x1500d0 (0xc16 bytes) 000204[1] 1734115585.050162 mapread 0x412dcd thru 0x413f3d (0x1171 bytes) 000205[0] 1734115585.063394 mapwrite 0x24b919 thru 0x24c3f8 (0xae0 bytes) 000206[0] 1734115585.103057 write 0x9b435d thru 0x9b43bf (0x63 bytes) 000208[1] 1734115585.116285 write 0x5fb19f thru 0x5fc0df (0xf41 bytes) 000209[1] 1734115585.125981 read 0x530aa5 thru 0x531df4 (0x1350 bytes) 000210[0] 1734115585.136773 write 0x1e1d2a thru 0x1e36dd (0x19b4 bytes) 000211[0] 1734115585.147612 read 0x5ac4e4 thru 0x5acad1 (0x5ee bytes) 000212[1] 1734115585.182385 read 0x31ca95 thru 0x31df25 (0x1491 bytes) 000213[1] 1734115585.230441 mapwrite 0x90a647 thru 0x90be90 (0x184a bytes) 000218[0] 1734115585.328079 mapwrite 0x3a1416 thru 0x3a19a4 (0x58f bytes) 000221[0] 1734115585.386880 write 0x7d14d7 thru 0x7d2ebf (0x19e9 bytes) 000222[1] 1734115585.399675 mapwrite 0x46167e thru 0x461a5e (0x3e1 bytes) 000223[1] 1734115585.441171 mapread 0x2ba2cf thru 0x2bae35 (0xb67 bytes) 000224[0] 1734115585.463771 write 0x13573b thru 0x136ef2 (0x17b8 bytes) 000228[1] 1734115585.507440 mapread 0x93eb4e thru 0x940160 (0x1613 bytes) 000230[0] 1734115585.529367 mapread 0x50da35 thru 0x50f0ca (0x1696 bytes) 000233[1] 1734115585.544105 write 0x6ff934 thru 0x700bae (0x127b bytes) 000234[1] 1734115585.551928 write 0x15507f thru 0x155ac0 (0xa42 bytes) 000235[0] 1734115585.563058 read 0x331bca thru 0x331fd1 (0x408 bytes) 000237[0] 1734115585.612275 mapread 0x45cba9 thru 0x45e5fc (0x1a54 bytes) 000239[1] 1734115585.649448 write 0x78b6d0 thru 0x78c950 (0x1281 bytes) 000241[1] 1734115585.661847 trunc from 0x9c806f to 0x1252c8 (0x8a2da6 bytes) 000242[0] 1734115585.675437 trunc from 0x1252c9 to 0x97898a (0x8536c2 bytes) 000243[1] 1734115585.696799 mapread 0x1be36e thru 0x1bfa4a (0x16dd bytes) 000244[1] 1734115585.712913 trunc from 0x97898b to 0x155b3d (0x822e4d bytes) 000245[0] 1734115585.734581 write 0x0a0217 thru 0x0a129a (0x1084 bytes) 000246[1] 1734115585.750660 write 0x9f498b thru 0x9f5a36 (0x10ac bytes) 000250[0] 1734115585.784828 read 0x04a00a thru 0x04a3d9 (0x3d0 bytes) 000252[0] 1734115585.826755 read 0x122c1a thru 0x12370b (0xaf2 bytes) 000253[0] 1734115585.834015 trunc from 0x9f5a37 to 0x7cd533 (0x228503 bytes) 000254[1] 1734115585.849929 read 0x32218d thru 0x323f08 (0x1d7c bytes) 000255[0] 1734115585.883598 write 0x2fbfaf thru 0x2fd7bb (0x180d bytes) 000257[0] 1734115585.936151 mapread 0x17dfb9 thru 0x17fd37 (0x1d7f bytes) 000258[0] 1734115585.936777 mapread 0x2209be thru 0x221057 (0x69a bytes) 000261[0] 1734115585.943822 mapwrite 0x699006 thru 0x69ad2e (0x1d29 bytes) 000265[0] 1734115586.025929 read 0x1309fa thru 0x132340 (0x1947 bytes) 000266[1] 1734115586.038360 read 0x072245 thru 0x072cf9 (0xab5 bytes) 000268[0] 1734115586.081394 mapread 0x2b4719 thru 0x2b5655 (0xf3d bytes) 000273[1] 1734115586.136706 mapread 0x14252e thru 0x14306c (0xb3f bytes) 000274[0] 1734115586.150190 read 0x43a157 thru 0x43adf6 (0xca0 bytes) 000276[1] 1734115586.150881 read 0x203c80 thru 0x204d8d (0x110e bytes) 000279[1] 1734115586.157140 trunc from 0x7cd534 to 0x82b0c6 (0x5db93 bytes) 000280[1] 1734115586.188036 read 0x2fc1d4 thru 0x2fd990 (0x17bd bytes) 000283[1] 1734115586.250581 trunc from 0x82b0c7 to 0x61ee63 (0x20c263 bytes) 000284[0] 1734115586.265180 trunc from 0x61ee64 to 0x93b312 (0x31c4af bytes) 000285[0] 1734115586.295670 mapread 0x29f025 thru 0x29f4b8 (0x494 bytes) 000286[0] 1734115586.326821 trunc from 0x93b313 to 0x357d3e (0x5e35d4 bytes) 000288[0] 1734115586.338673 trunc from 0x357d3f to 0x599663 (0x241925 bytes) 000292[1] 1734115586.365683 read 0x1e2f13 thru 0x1e3b6f (0xc5d bytes) 000295[1] 1734115586.437705 trunc from 0x599664 to 0x7d0eab (0x237848 bytes) 000296[1] 1734115586.456045 mapwrite 0x21157e thru 0x213149 (0x1bcc bytes) 000297[1] 1734115586.516682 mapread 0x1debfc thru 0x1e0b45 (0x1f4a bytes) 000298[0] 1734115586.526743 write 0x71fd4d thru 0x7217fc (0x1ab0 bytes) 000300[1] 1734115586.574437 mapwrite 0x693399 thru 0x6936cb (0x333 bytes) 000303[1] 1734115586.615257 read 0x0f8dcb thru 0x0f97d3 (0xa09 bytes) 000304[0] 1734115586.650572 mapread 0x40a7ad thru 0x40c261 (0x1ab5 bytes) 000305[0] 1734115586.687368 mapread 0x1f4b65 thru 0x1f6997 (0x1e33 bytes) 000306[1] 1734115586.706648 mapread 0x6bfdb4 thru 0x6c186f (0x1abc bytes) 000307[1] 1734115586.714075 write 0x52ea2e thru 0x52f0cf (0x6a2 bytes) 000308[0] 1734115586.731187 mapread 0x6e00d3 thru 0x6e0c8c (0xbba bytes) 000314[1] 1734115586.748992 read 0x06ee52 thru 0x07048d (0x163c bytes) 000318[0] 1734115586.793173 mapwrite 0x96962d thru 0x96a13e (0xb12 bytes) 000321[1] 1734115586.881857 write 0x48b2a7 thru 0x48b75e (0x4b8 bytes) 000324[0] 1734115586.908586 read 0x8edcb1 thru 0x8ee43c (0x78c bytes) 000326[1] 1734115586.920294 trunc from 0x96a13f to 0x6f41b7 (0x275f87 bytes) 000328[0] 1734115586.944844 mapread 0x6bd126 thru 0x6bd2be (0x199 bytes) 000330[0] 1734115586.970119 trunc from 0x6f41b8 to 0x8e100c (0x1ece55 bytes) 000331[0] 1734115586.989710 mapwrite 0x4271c1 thru 0x42773a (0x57a bytes) 000332[1] 1734115587.047490 read 0x4e3a50 thru 0x4e3f5c (0x50d bytes) 000334[1] 1734115587.064646 write 0x7c918f thru 0x7ca3fb (0x126d bytes) 000339[1] 1734115587.099142 read 0x42ce64 thru 0x42d2e4 (0x481 bytes) 000342[0] 1734115587.124777 mapwrite 0x795f53 thru 0x796d7f (0xe2d bytes) 000346[0] 1734115587.208010 trunc from 0x8e100d to 0x76b703 (0x175909 bytes) 000347[1] 1734115587.218077 mapread 0x5d76c7 thru 0x5d93f4 (0x1d2e bytes) 000350[0] 1734115587.244189 write 0x3ec851 thru 0x3ed58c (0xd3c bytes) 000351[0] 1734115587.250609 trunc from 0x76b704 to 0x7ae08c (0x42989 bytes) 000357[1] 1734115587.279336 trunc from 0x7ae08d to 0x19d120 (0x610f6c bytes) 000360[1] 1734115587.302249 mapread 0x06f938 thru 0x070920 (0xfe9 bytes) 000361[1] 1734115587.331718 trunc from 0x19d121 to 0x397174 (0x1fa054 bytes) 000363[0] 1734115587.351298 read 0x01b359 thru 0x01bce1 (0x989 bytes) 000365[1] 1734115587.398645 trunc from 0x397175 to 0x3647fe (0x32976 bytes) 000367[0] 1734115587.423856 read 0x2c3df5 thru 0x2c5a46 (0x1c52 bytes) 000371[0] 1734115587.480823 trunc from 0x3647ff to 0x3779a7 (0x131a9 bytes) 000372[0] 1734115587.516709 write 0x260c08 thru 0x2612a5 (0x69e bytes) 000376[0] 1734115587.552061 mapwrite 0x88fe20 thru 0x891db0 (0x1f91 bytes) 000379[0] 1734115587.629738 mapread 0x68a0a1 thru 0x68bd58 (0x1cb8 bytes) 000385[0] 1734115587.655157 mapwrite 0x546a5f thru 0x547cf4 (0x1296 bytes) 000391[0] 1734115587.701746 read 0x4c43eb thru 0x4c4dbc (0x9d2 bytes) 000393[1] 1734115587.712185 mapwrite 0x0f376c thru 0x0f4746 (0xfdb bytes) 000394[1] 1734115587.767742 write 0x83b0c9 thru 0x83c06d (0xfa5 bytes) 000397[1] 1734115587.795195 read 0x539f7c thru 0x53bb68 (0x1bed bytes) 000399[1] 1734115587.851402 write 0x718bc9 thru 0x71a7ae (0x1be6 bytes) 000400[1] 1734115587.872138 mapwrite 0x215c10 thru 0x216f57 (0x1348 bytes) 000401[1] 1734115587.912891 mapwrite 0x237832 thru 0x2386e7 (0xeb6 bytes) 000405[1] 1734115587.961567 mapwrite 0x25aa7c thru 0x25b390 (0x915 bytes) 000406[0] 1734115587.995245 trunc from 0x891db1 to 0x1fb9ec (0x6963c4 bytes) 000407[0] 1734115588.018639 mapread 0x17c220 thru 0x17cfb4 (0xd95 bytes) 000409[0] 1734115588.052633 write 0x5fe6b5 thru 0x600500 (0x1e4c bytes) 000410[0] 1734115588.054046 write 0x3962ca thru 0x3964d7 (0x20e bytes) 000414[0] 1734115588.083655 mapread 0x5226be thru 0x523c11 (0x1554 bytes) 000417[1] 1734115588.104266 trunc from 0x600501 to 0x319913 (0x2e6bed bytes) 000418[0] 1734115588.129846 write 0x7ac81e thru 0x7ae583 (0x1d66 bytes) 000419[1] 1734115588.144119 mapread 0x3bd294 thru 0x3be00a (0xd77 bytes) 000422[1] 1734115588.185086 read 0x684568 thru 0x684fad (0xa46 bytes) 000424[0] 1734115588.228200 mapwrite 0x444eb3 thru 0x446065 (0x11b3 bytes) 000425[1] 1734115588.280221 write 0x18bbe0 thru 0x18d49e (0x18bf bytes) 000430[0] 1734115588.309915 mapwrite 0x7cd6b1 thru 0x7cdf84 (0x8d4 bytes) 000431[0] 1734115588.354357 mapwrite 0x71aed7 thru 0x71bebf (0xfe9 bytes) 000436[0] 1734115588.400947 mapread 0x4e0fd4 thru 0x4e148f (0x4bc bytes) 000437[1] 1734115588.408651 mapread 0x315016 thru 0x316e9d (0x1e88 bytes) 000438[1] 1734115588.435603 trunc from 0x7cdf85 to 0x364c9a (0x4692ea bytes) 000439[1] 1734115588.464236 write 0x706ef9 thru 0x707c1d (0xd25 bytes) 000440[1] 1734115588.468599 write 0x757d1e thru 0x7581a3 (0x486 bytes) 000443[1] 1734115588.482566 read 0x71ece6 thru 0x71ff8c (0x12a7 bytes) 000444[0] 1734115588.509559 mapwrite 0x1e4e41 thru 0x1e5eb5 (0x1075 bytes) 000445[1] 1734115588.561225 write 0x23ab7f thru 0x23b2f9 (0x77b bytes) 000446[0] 1734115588.569486 mapwrite 0x9fceb5 thru 0x9fe315 (0x1461 bytes) 000449[0] 1734115588.648722 read 0x3945b2 thru 0x3961d8 (0x1c27 bytes) 000455[1] 1734115588.712392 read 0x844924 thru 0x846790 (0x1e6d bytes) 000456[0] 1734115588.753254 read 0x21fdc1 thru 0x21fe3c (0x7c bytes) 000457[0] 1734115588.788586 mapwrite 0x4c7081 thru 0x4c9060 (0x1fe0 bytes) 000458[0] 1734115588.812529 trunc from 0x9fe316 to 0x9c1fba (0x3c35b bytes) 000463[1] 1734115588.850249 read 0x0462be thru 0x047b47 (0x188a bytes) 000466[1] 1734115588.892779 write 0x4b8821 thru 0x4b916f (0x94f bytes) 000468[0] 1734115588.932646 mapwrite 0x1751e4 thru 0x17640f (0x122c bytes) 000470[0] 1734115588.984643 write 0x6fd09c thru 0x6fe63a (0x159f bytes) 000473[1] 1734115589.040538 read 0x58c043 thru 0x58ddc4 (0x1d82 bytes) 000476[1] 1734115589.075870 trunc from 0x9c1fbb to 0x23cbef (0x7853cb bytes) 000478[1] 1734115589.094337 read 0x1516ba thru 0x152f6f (0x18b6 bytes) 000480[0] 1734115589.115565 read 0x1d8f99 thru 0x1d928b (0x2f3 bytes) 000484[1] 1734115589.163803 write 0x16f2fa thru 0x16fe74 (0xb7b bytes) 000485[0] 1734115589.172958 mapwrite 0x35a240 thru 0x35a297 (0x58 bytes) 000488[0] 1734115589.250958 trunc from 0x35a298 to 0x7317bb (0x3d7524 bytes) 000493[0] 1734115589.279550 trunc from 0x7317bc to 0x609e63 (0x127958 bytes) 000494[1] 1734115589.293888 write 0x42bb68 thru 0x42d4be (0x1957 bytes) 000496[0] 1734115589.314774 write 0x53bcb8 thru 0x53c2f2 (0x63b bytes) 000498[1] 1734115589.359419 mapwrite 0x772782 thru 0x772d3f (0x5be bytes) 000500[0] 1734115589.416390 trunc from 0x772d40 to 0x886431 (0x1136f2 bytes) All operations completed A-OK! PASS 16b (10s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 13:46:30 (1734115590) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 13:46:32 (1734115592) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.194025 s, 43.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.164776 s, 50.9 MB/s ldlm.namespaces.MGC192.168.202.104@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b54d5800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b6d7d000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b54d5800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b6d7d000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b54d5800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b6d7d000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.38344 s, 23.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.04445 s, 100 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.68531 s, 153 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.98901 s, 52.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.23399 s, 24.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.926663 s, 113 MB/s PASS 16d (17s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 13:46:49 (1734115609) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0679555 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539806 s, 19.4 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 13:46:51 (1734115611) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 13:47:13 (1734115633) TIMEOUT PASS 16g (22s) == sanityn test 16h: mmap read after truncate file ======= 13:47:35 (1734115655) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) == sanityn test 16i: read after truncate file ============ 13:47:37 (1734115657) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 16j: race dio with buffered i/o ========== 13:47:39 (1734115659) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.392235 s, 66.8 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.300657 s, 87.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.831769 s, 31.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.178244 s, 147 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.725624 s, 36.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.178984 s, 146 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.830042 s, 31.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.293096 s, 89.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.367376 s, 71.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.869805 s, 30.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.920409 s, 28.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.438292 s, 59.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.450507 s, 58.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.736855 s, 35.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.780325 s, 33.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.294272 s, 89.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.386987 s, 67.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.764122 s, 34.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.855179 s, 30.7 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (10s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 13:47:49 (1734115669) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c skipping zero size read truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! + fsx 1 done: rc=0 Fri Dec 13 13:48:12 EST 2024 All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Fri Dec 13 13:48:14 EST 2024 + fsx 3 done: rc=0 Fri Dec 13 13:48:14 EST 2024 All operations completed A-OK! + fsx 4 done: rc=0 Fri Dec 13 13:48:14 EST 2024 All operations completed A-OK! + fsx 5 done: rc=0 Fri Dec 13 13:48:14 EST 2024 PASS 16k (30s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 13:48:19 (1734115699) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 13:48:23 (1734115703) mmap test1: basic mmap operation (PASS, 0.023533s) mmap test2: MAP_PRIVATE not write back (PASS, 0.019482s) mmap test3: concurrent mmap ops on two nodes (PASS, 4.496s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.0816s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.029257s) mmap test6: check mmap write/read content on two nodes (PASS, 0.044853s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.31973s) mmap test11: multi-thread shared mmap (PASS, 0.30006s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0125s) PASS 18 (12s) == sanityn test 19: test concurrent uncached read races ========================================================================= 13:48:35 (1734115715) oleg204-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg204-client: oleg204-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 13:48:37 (1734115717) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 13:48:38 (1734115718) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 13:48:40 (1734115720) now is 1734115720 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 new atime is 1734115781 PASS 23 (62s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 13:49:42 (1734115782) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 33.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 39.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118066 402 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117199 399 116800 1% /mnt/lustre2[OST:1] filesystem_summary: 234851 387 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118066 402 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117199 399 116800 1% /mnt/lustre2[OST:1] filesystem_summary: 234851 387 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 13:49:44 (1734115784) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 13:49:46 (1734115786) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 13:49:47 (1734115787) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 13:49:48 (1734115788) utime: good mknod times 1734115787 <= 1734115788 <= 1734115788 for /mnt/lustre/f26a utime: good mknod times 1734115787 <= 1734115788 <= 1734115788 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 13:49:50 (1734115790) PASS 26b (4s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 13:49:54 (1734115794) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2024-12-13 13:49:54.000000000 -0500 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2024-12-13 13:49:54.000000000 -0500 Birth: - PASS 26c (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 13:49:57 (1734115797) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 13:50:01 (1734115801) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 13:50:04 (1734115804) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg204-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.104@tcp:/lustre 7542784 39936 7496704 1% /mnt/lustre2 PASS 31b (18s) == sanityn test 31r: open-rename(replace) race =========== 13:50:21 (1734115821) -rw-r--r-- 1 root root 0 Dec 13 13:50 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 13:50:25 (1734115825) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13511 Terminated tail -f $tfile PASS 31s (2s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 13:50:27 (1734115827) PASS 31t (2s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 13:50:30 (1734115830) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 13:50:31 (1734115831) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 13:50:31 (1734115831) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 13:50:32 (1734115832) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 13:50:33 (1734115833) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b54d5800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b54d5800.ost_server_uuid in FULL state after 0 sec oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b54d5800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b54d5800.ost_server_uuid in IDLE state after 0 sec oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b54d5800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b54d5800.ost_server_uuid in FULL state after 0 sec oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b54d5800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b54d5800.ost_server_uuid in IDLE state after 0 sec oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b54d5800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b54d5800.ost_server_uuid in FULL state after 0 sec oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b54d5800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b54d5800.ost_server_uuid in IDLE state after 0 sec PASS 34 (87s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 13:52:00 (1734115920) fail_loc=0x80000317 Race attempt 0 Wait for 20140 20284 for 60 sec... total: 4000 open/close in 9.46 seconds: 422.97 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.104@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre 192.168.202.104@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 13:53:03 (1734115983) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg204-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.268191 s, 195 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.254993 s, 206 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7498752) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.250747 s, 209 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7494656) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.27874 s, 188 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.276385 s, 190 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7496704) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.281843 s, 186 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.246302 s, 213 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.261709 s, 200 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.261762 s, 200 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.281877 s, 186 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7496704) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.284476 s, 184 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7498752) after_dd(7447552) after(7498752) PASS 36 (227s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 13:56:50 (1734116210) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 - create 9671 (time 1734116221.25 total 10.00 last 967.07) total: 10000 create in 10.33 seconds: 968.16 ops/second PASS 37 (13s) == sanityn test 39a: file mtime does not change after rename ========================================================== 13:57:03 (1734116223) repeat after cancel_lru_locks PASS 39a (1s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 13:57:04 (1734116224) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 13:57:07 (1734116227) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 13:57:10 (1734116230) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 13:57:11 (1734116231) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 13:57:18 (1734116238) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 13:57:25 (1734116245) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 13:57:32 (1734116252) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 13:57:39 (1734116259) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 13:57:46 (1734116266) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 13:57:50 (1734116270) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 13:57:54 (1734116274) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 13:57:58 (1734116278) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 13:58:03 (1734116283) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 13:58:07 (1734116287) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 13:58:11 (1734116291) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 13:58:15 (1734116295) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 13:58:20 (1734116300) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (647s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 14:09:07 (1734116947) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 14:09:12 (1734116952) open: File exists Conflict PASS 42b (6s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 14:09:18 (1734116958) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 14:09:23 (1734116963) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 14:09:28 (1734116968) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (6s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 14:09:34 (1734116974) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 14:09:39 (1734116979) Conflict PASS 42g (6s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 14:09:45 (1734116985) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 14:09:50 (1734116990) PASS 43a (21s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 14:10:11 (1734117011) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 14:10:16 (1734117016) Conflict PASS 43c (5s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 14:10:21 (1734117021) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 14:10:26 (1734117026) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 14:10:31 (1734117031) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 14:10:36 (1734117036) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 14:10:41 (1734117041) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 14:10:46 (1734117046) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 14:10:47 (1734117047) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (39s) == sanityn test 43k: unlink vs create ==================== 14:11:26 (1734117086) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (409s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 14:18:15 (1734117495) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Dec 13 14:18:19 EST 2024 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 14:18:20 (1734117500) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 14:18:24 (1734117504) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 14:18:29 (1734117509) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 14:18:33 (1734117513) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 14:18:38 (1734117518) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 14:18:42 (1734117522) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 14:18:47 (1734117527) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 14:18:51 (1734117531) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 14:18:52 (1734117532) PASS 45a (23s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 14:19:15 (1734117555) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 14:19:20 (1734117560) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 14:19:24 (1734117564) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 14:19:28 (1734117568) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 14:19:33 (1734117573) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 14:19:37 (1734117577) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 14:19:41 (1734117581) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 14:19:46 (1734117586) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 14:19:46 (1734117586) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (360s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 14:25:46 (1734117946) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 14:25:51 (1734117951) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 14:25:55 (1734117955) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 14:25:59 (1734117959) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 14:26:04 (1734117964) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 14:26:08 (1734117968) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 14:26:12 (1734117972) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 14:26:16 (1734117976) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 14:26:21 (1734117981) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 14:26:22 (1734117982) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (0s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 14:26:22 (1734117982) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 14:26:23 (1734117983) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 14:26:24 (1734117984) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 14:26:25 (1734117985) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (0s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 14:26:25 (1734117985) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 14:26:26 (1734117986) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 14:26:27 (1734117987) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00457373 s, 2.2 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 14:26:34 (1734117994) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00343975 s, 46.2 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3518: kill: (32019) - No such process PASS 51a (3s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 14:26:37 (1734117997) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00301765 s, 339 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145873 s, 71.9 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 14:26:51 (1734118011) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00442917 s, 231 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 14:26:57 (1734118017) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982287 s, 107 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 14:26:59 (1734118019) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 14:27:03 (1734118023) llite.lustre-ffff8800b54d5800.xattr_cache=1 llite.lustre-ffff8800b6d7d000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:27:03.000000000 -0500 Modify: 2024-12-13 14:27:03.000000000 -0500 Change: 2024-12-13 14:27:03.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:27:09.000000000 -0500 Modify: 2024-12-13 14:27:09.000000000 -0500 Change: 2024-12-13 14:27:09.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:27:14.000000000 -0500 Modify: 2024-12-13 14:27:14.000000000 -0500 Change: 2024-12-13 14:27:14.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:27:14.000000000 -0500 Modify: 2024-12-13 14:27:14.000000000 -0500 Change: 2024-12-13 14:27:14.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:27:20.000000000 -0500 Modify: 2024-12-13 14:27:20.000000000 -0500 Change: 2024-12-13 14:27:20.000000000 -0500 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:27:20.000000000 -0500 Modify: 2024-12-13 14:27:20.000000000 -0500 Change: 2024-12-13 14:27:20.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 14:27:26 (1734118046) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 14:27:33 (1734118053) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 14:27:40 (1734118060) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 14:27:52 (1734118072) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 14:28:01 (1734118081) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1734118081 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 3.39106 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (19s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 14:28:20 (1734118100) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1734118101 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.52127 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (31s) == sanityn test 60: Verify data_version behaviour ======== 14:28:51 (1734118131) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 14:28:53 (1734118133) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 14:28:54 (1734118134) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 14:28:56 (1734118136) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00363526 s, 18.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 14:28:57 (1734118137) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00355463 s, 18.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 14:28:58 (1734118138) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 14:28:59 (1734118139) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00347875 s, 18.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 14:29:00 (1734118140) llite.lustre-ffff8800b54d5800.xattr_cache=1 llite.lustre-ffff8800b6d7d000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 14:29:01 (1734118141) llite.lustre-ffff8800b54d5800.xattr_cache=1 llite.lustre-ffff8800b6d7d000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 14:29:03 (1734118143) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00231696 s, 442 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00215615 s, 475 kB/s 12050: get lock2 12051: get lock1 12051: done 12051: get lock2 12050: done 12050: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 12051: cannot get lock2: Resource deadlock avoided 12051: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 12050: cannot get lock1: Resource deadlock avoided 12050: done 12050: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 14:29:09 (1734118149) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.012325 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933369 s, 112 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4107: 12691 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 14:29:15 (1734118155) opening files: ........ closing files: ........ PASS 76 (14s) == sanityn test 77a: check FIFO NRS policy =============== 14:29:29 (1734118169) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0904494 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900398 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133997 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208986 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163516 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125316 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291131 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222428 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310963 s, 33.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327271 s, 32.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0363033 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323444 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366809 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308187 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339456 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340457 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413035 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072567 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714856 s, 147 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0119652 s, 87.6 MB/s , 0.00763134 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870892 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944016 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882412 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959857 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138223 s, 75.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125528 s, 83.5 MB/s , 0.011068 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125217 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011631 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136154 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130615 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147174 s, 71.2 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 14:29:31 (1734118171) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.086471 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882772 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108351 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112415 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157003 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00716785 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187086 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153428 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00910582 s, 115 MB/s 1048576 bytes (1.0 MB) copied, 0.0117679 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340889 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253249 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333569 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375144 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212635 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316656 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043675 s, 24.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00506833 s, 207 MB/s , 0.00512074 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684282 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977557 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0084883 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.00693418 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00687107 s, 153 MB/s , 0.00508205 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010253 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815219 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874226 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786529 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787448 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899843 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010862 s, 96.5 MB/s , 0.00920733 s, 114 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0770467 s, 218 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0104982 s, 99.9 MB/s , 0.0104095 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012039 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226476 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255679 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315035 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278685 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284287 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371648 s, 28.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0390693 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331636 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436768 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389274 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391307 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472996 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0558049 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007066 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00558646 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670595 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00536762 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843203 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999458 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833906 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00717326 s, 146 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104268 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801806 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0082828 s, 127 MB/s 1048576 bytes (1.0 MB) copied, 0.00875758 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935993 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994246 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104002 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00949697 s, 110 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 14:29:34 (1734118174) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0799626 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984466 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123904 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011498 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341115 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337318 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365837 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449015 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469952 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.04477 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486644 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426043 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354559 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037979 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366072 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461205 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444371 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00591004 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00853901 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697727 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00653808 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884291 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829398 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847358 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766455 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0080258 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814939 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0082911 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085538 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100772 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00863987 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961413 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118254 s, 88.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0864576 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00744693 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00604303 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911795 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00736895 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907768 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108476 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115945 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00778028 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153493 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562787 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501787 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602877 s, 17.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603179 s, 17.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0539672 s, 19.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0535579 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489432 s, 21.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00625456 s, 168 MB/s , 0.00631534 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069599 s, 151 MB/s , 0.006991 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755157 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00801046 s, 131 MB/s 1048576 bytes (1.0 MB) copied, 0.00790145 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897604 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899532 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182884 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107686 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108616 s, 96.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0214462 s, 48.9 MB/s , 0.0114109 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125868 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144913 s, 72.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0839098 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826796 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110637 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159783 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168241 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108158 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143874 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121683 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277977 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272465 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328022 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190824 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181376 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394438 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236872 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355617 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425816 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00519485 s, 202 MB/s 1048576 bytes (1.0 MB) copied, 0.00518027 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150375 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00556456 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673239 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155231 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00703353 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00729915 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00777598 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00875394 s, 120 MB/s , 0.00870348 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944133 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954378 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914631 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085446 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113663 s, 92.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (3s) == sanityn test 77d: check TRR nrs policy ================ 14:29:37 (1734118177) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0839585 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763918 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00754727 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268631 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291394 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323653 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032344 s, 32.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269792 s, 38.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0290991 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341851 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382857 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321525 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037851 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039003 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0359995 s, 29.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0373104 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492888 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00688311 s, 152 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0087657 s, 120 MB/s , 0.00621447 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00636987 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901594 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766767 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00721885 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00490292 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937797 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089411 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955168 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921705 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959138 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097562 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115896 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116259 s, 90.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0860997 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959792 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109462 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271948 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239505 s, 43.8 MB/s , 0.0254432 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288143 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263239 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297127 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282795 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0648349 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0696131 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0664403 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0711878 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0693581 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0725567 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0731608 s, 14.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899644 s, 117 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00611979 s, 171 MB/s , 0.00600636 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691417 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00716942 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102354 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822119 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161749 s, 648 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897872 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109928 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105285 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00989793 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986138 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106354 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123911 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134277 s, 78.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0736638 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978718 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138164 s, 75.9 MB/s 1048576 bytes (1.0 MB) copied, 0.014 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252712 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351424 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371376 s, 28.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411544 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433422 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420622 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411505 s, 25.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0445701 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446617 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427065 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443663 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449701 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0499857 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00575179 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00598974 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00648939 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837111 s, 125 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00774838 s, 135 MB/s 1048576 bytes (1.0 MB) copied, 0.00782505 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771342 s, 136 MB/s , 0.00741812 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00119572 s, 877 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897652 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089988 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905876 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915611 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962882 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893795 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105879 s, 99.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 14:29:41 (1734118181) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.4@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0825616 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951476 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797531 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015323 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00647137 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015633 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152213 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283185 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117801 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136236 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120453 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290227 s, 36.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343973 s, 30.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0361444 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388163 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362827 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207932 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963163 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00664843 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901656 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879774 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116996 s, 89.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971394 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00917151 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973069 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105229 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110104 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119986 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113301 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120665 s, 86.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0138406 s, 75.8 MB/s , 0.0130056 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125299 s, 83.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0799751 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940314 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131555 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237817 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313232 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381561 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385617 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269859 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410024 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453792 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408124 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041561 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416804 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474865 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424975 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450029 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042373 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00829605 s, 126 MB/s , 0.00756406 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943744 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00789898 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00746203 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814358 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855465 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772337 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872739 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00818595 s, 128 MB/s 1048576 bytes (1.0 MB) copied, 0.0081755 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918441 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103773 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00870589 s, 120 MB/s , 0.0090452 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112985 s, 92.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0903009 s, 186 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00642327 s, 163 MB/s 1048576 bytes (1.0 MB) copied, 0.00776598 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095375 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088497 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210165 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121114 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151076 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912097 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217675 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390911 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411874 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388337 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397032 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456399 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357599 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055659 s, 18.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00709243 s, 148 MB/s , 0.0161482 s, 64.9 MB/s , 0.0148815 s, 70.5 MB/s , 0.00662856 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885763 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901466 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802483 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877966 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908412 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168391 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103182 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.020364 s, 51.5 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105313 s, 99.6 MB/s , 0.00971673 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114163 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115867 s, 90.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.082017 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115433 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105314 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119227 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170514 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282312 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232935 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448433 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381672 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355677 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477258 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470225 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552368 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516246 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0559208 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412953 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0564112 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769508 s, 136 MB/s , 0.00516041 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0018956 s, 553 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697846 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731249 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00444163 s, 236 MB/s , 0.0110026 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0113696 s, 92.2 MB/s , 0.00930684 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923337 s, 114 MB/s , 0.00880673 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111567 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0101239 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0102128 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010357 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115268 s, 91.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 14:29:46 (1734118186) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.095551 s, 176 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0074063 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670538 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010895 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940344 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122937 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02151 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241775 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152396 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265431 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260508 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353155 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368418 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295093 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299972 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031757 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370533 s, 28.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00640969 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00642957 s, 163 MB/s , 0.00824858 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680784 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107902 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100002 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871581 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00160077 s, 655 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104679 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108223 s, 96.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0107048 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973712 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105139 s, 99.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0095001 s, 110 MB/s , 0.0104949 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115746 s, 90.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.086615 s, 194 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00724433 s, 145 MB/s 1048576 bytes (1.0 MB) copied, 0.00729587 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114163 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00628252 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152292 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025597 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200134 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355819 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357813 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281856 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276794 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305253 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432768 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301425 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361067 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385953 s, 27.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847733 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00700796 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010594 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805354 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835779 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107484 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090391 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105095 s, 99.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100547 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010251 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115622 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095145 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972006 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100222 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0112161 s, 93.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0112634 s, 93.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0965236 s, 174 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982045 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133375 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328048 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347885 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352771 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380166 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395737 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045883 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468138 s, 22.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476773 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520231 s, 20.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0508298 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490377 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514948 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492181 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0564788 s, 18.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817823 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00761038 s, 138 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00758948 s, 138 MB/s , 0.00766912 s, 137 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114609 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00169113 s, 620 MB/s 1048576 bytes (1.0 MB) copied, 0.00901419 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932851 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105915 s, 99.0 MB/s 1048576 bytes (1.0 MB) copied, 0.00995495 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106147 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109695 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00989772 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102082 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105147 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134051 s, 78.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0938659 s, 179 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112145 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120894 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338311 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389631 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306303 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433591 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352777 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445202 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045539 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457149 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430037 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477116 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482842 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425097 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449657 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502652 s, 20.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00720416 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0054035 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00572273 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00584701 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712834 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100892 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804758 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861857 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088931 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0057203 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854414 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102596 s, 102 MB/s , 0.0080455 s, 130 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00976057 s, 107 MB/s , 0.00973893 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110949 s, 94.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 14:29:52 (1734118192) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (2s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 14:29:54 (1734118194) oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg204-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg204-client: oleg204-server: ssh exited with exit code 2 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg204-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg204-client: oleg204-server: ssh exited with exit code 12 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg204-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg204-client: oleg204-server: ssh exited with exit code 12 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg204-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg204-client: oleg204-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 14:29:57 (1734118197) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 14:30:03 (1734118203) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0846548 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109916 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014414 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202309 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253669 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222328 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315864 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205158 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169039 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306944 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333547 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323858 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396166 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370287 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318898 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414076 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431598 s, 24.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00201396 s, 521 MB/s 1048576 bytes (1.0 MB) copied, 0.00675794 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083393 s, 126 MB/s , 0.00662038 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973818 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00713596 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010358 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816645 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00749252 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797041 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883488 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881923 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864217 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997096 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953781 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106797 s, 98.2 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92213 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4408 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 14:30:53 (1734118253) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.21924 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116257 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142183 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265145 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249201 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321611 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324328 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215161 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365584 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.172247 s, 6.1 MB/s , 0.174704 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.182195 s, 5.8 MB/s , 0.183764 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178986 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180414 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191336 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18999 s, 5.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0123273 s, 85.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0137096 s, 76.5 MB/s , 0.012507 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137114 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133939 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128774 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158429 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015531 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016374 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162883 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168251 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163841 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164506 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170853 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173723 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019976 s, 52.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0406 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9633 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22339 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132016 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179403 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0187666 s, 55.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0171706 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231889 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268413 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018443 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162233 s, 64.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.162721 s, 6.4 MB/s , 0.162868 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.172083 s, 6.1 MB/s , 0.175062 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.189697 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190382 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182226 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187598 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100035 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110013 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107916 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106134 s, 98.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0114357 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133006 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119563 s, 87.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0123022 s, 85.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0154188 s, 68.0 MB/s , 0.0130056 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0148525 s, 70.6 MB/s , 0.0149849 s, 70.0 MB/s 1+0 records in 1+0 records out , 0.0140273 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153955 s, 68.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0167684 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177537 s, 59.1 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8549 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9511 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (133s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 14:33:05 (1734118385) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.439575 s, 38.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118967 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133631 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137281 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177832 s, 59.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0375859 s, 27.9 MB/s , 0.0237921 s, 44.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0302792 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387473 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155062 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15573 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.14718 s, 7.1 MB/s , 0.166503 s, 6.3 MB/s , 0.140934 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153769 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174884 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158609 s, 6.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136079 s, 77.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0134818 s, 77.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0141603 s, 74.1 MB/s , 0.0133649 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131601 s, 79.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0157125 s, 66.7 MB/s , 0.0153938 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146017 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014068 s, 74.5 MB/s 1+0 records in 1+0 records out , 0.0175112 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143304 s, 73.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0147536 s, 71.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153954 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135242 s, 77.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0160225 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184254 s, 56.9 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8394 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9626 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg204-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.446315 s, 37.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166218 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182537 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175668 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194357 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274224 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289911 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156538 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121983 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142767 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142885 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137231 s, 7.6 MB/s 1048576 bytes (1.0 MB) copied, 0.152744 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153162 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154259 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164566 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167657 s, 6.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971943 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100385 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100893 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121231 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108466 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010919 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124331 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116764 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146135 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141186 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143441 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0154687 s, 67.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0144928 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150678 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016375 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185058 s, 56.7 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0305 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9521 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (130s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 14:35:15 (1734118515) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.112195 s, 150 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913434 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106634 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136437 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242518 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142265 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026727 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250569 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279417 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104968 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308036 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00698622 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249998 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346189 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0340959 s, 30.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0299142 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882715 s, 119 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124091 s, 84.5 MB/s , 0.0114624 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119681 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110201 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111946 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109148 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00188048 s, 558 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934472 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103057 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01017 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0132619 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115462 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108761 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107803 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116507 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142389 s, 73.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94046 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73647 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.4@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.4@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0960925 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00850155 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117767 s, 89.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243801 s, 43.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0282101 s, 37.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.029128 s, 36.0 MB/s , 0.0283649 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261436 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033274 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307248 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309053 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260914 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288516 s, 36.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.220158 s, 4.8 MB/s , 0.210164 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.258809 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.264401 s, 4.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106646 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100834 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829296 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930346 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.00836184 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967512 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956237 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978014 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105083 s, 99.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107612 s, 97.4 MB/s , 0.010788 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109143 s, 96.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012873 s, 81.5 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0123072 s, 85.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0120124 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133805 s, 78.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92109 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73049 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.4@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0903721 s, 186 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076226 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135816 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133564 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289706 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380324 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0357369 s, 29.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0336052 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360099 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386479 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402418 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441065 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347548 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447689 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414731 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385871 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418795 s, 25.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00653418 s, 160 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00815629 s, 129 MB/s , 0.00574896 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919529 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010647 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923163 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759533 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775411 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892631 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905579 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927131 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993943 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999494 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00962658 s, 109 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101581 s, 103 MB/s , 0.00973634 s, 108 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92212 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87938 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.12148 s, 138 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109374 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987073 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292385 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030072 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347587 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343543 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333316 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301902 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360338 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441878 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039708 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447704 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105686 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101438 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112753 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103017 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971512 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121441 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116108 s, 90.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127157 s, 82.5 MB/s , 0.0109509 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150686 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145468 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136169 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124944 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115056 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128034 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122278 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128881 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131556 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139081 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141253 s, 74.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76958 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72943 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91075 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7297 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.124532 s, 135 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208245 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266564 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250397 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271537 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342306 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379581 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304912 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393608 s, 26.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0967769 s, 10.8 MB/s , 0.0871903 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.10542 s, 9.9 MB/s , 0.103099 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100524 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0921697 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122234 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127841 s, 8.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108626 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140057 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0117439 s, 89.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0134338 s, 78.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0116423 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133509 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133166 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141827 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140857 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142172 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141574 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132719 s, 79.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158895 s, 66.0 MB/s , 0.0156164 s, 67.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0145487 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165836 s, 63.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6544 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4323 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.108416 s, 155 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111141 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108609 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295198 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280811 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365932 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376795 s, 27.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389322 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357912 s, 29.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0364395 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356232 s, 29.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0348023 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352091 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043843 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429169 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389376 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0491983 s, 21.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137739 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0071153 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103601 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886219 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957631 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180747 s, 58.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0097286 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110282 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102416 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186888 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126155 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012707 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013082 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133249 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153817 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155148 s, 67.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94791 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73099 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0985161 s, 170 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165534 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017539 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200124 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253811 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315845 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319041 s, 32.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361242 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024881 s, 42.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0252654 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195851 s, 53.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0781105 s, 13.4 MB/s , 0.0684647 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.131738 s, 8.0 MB/s , 0.122031 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112579 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122243 s, 8.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00842487 s, 124 MB/s , 0.00817645 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861074 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738811 s, 142 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00814718 s, 129 MB/s , 0.0107198 s, 97.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971694 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0103156 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00580857 s, 181 MB/s 1048576 bytes (1.0 MB) copied, 0.00867371 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885417 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106945 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926838 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009634 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108697 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119098 s, 88.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77443 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73219 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7608 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7286 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (334s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 14:40:49 (1734118849) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 14:40:51 (1734118851) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg204-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.26765 s, 321 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg204-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.015 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 14:41:41 (1734118901) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0880324 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130131 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128424 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013635 s, 76.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103706 s, 101 MB/s , 0.0159039 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146272 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00722462 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787687 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00808744 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208073 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248317 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316076 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297016 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323756 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041334 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455395 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092325 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758576 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691551 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00588284 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00836885 s, 125 MB/s 1048576 bytes (1.0 MB) copied, 0.00946464 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805344 s, 130 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0100265 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.00804141 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848503 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.00920338 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888959 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899087 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00942811 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111237 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108061 s, 97.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91501 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96891 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.234911 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139086 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164802 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240267 s, 43.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288871 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356423 s, 29.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440238 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345383 s, 30.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0409831 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339321 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444805 s, 23.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0357479 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379591 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.274397 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.267692 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.315547 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314439 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241812 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233492 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255698 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244294 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248082 s, 42.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0242272 s, 43.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0253241 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251353 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251801 s, 41.6 MB/s , 0.0248627 s, 42.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0256301 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253767 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260185 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0267287 s, 39.2 MB/s 1048576 bytes (1.0 MB) copied, 0.02651 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277274 s, 37.8 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg204-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg204-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92929 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96697 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (60s) == sanityn test 77o: Changing rank should not panic ====== 14:42:41 (1734118961) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 14:42:44 (1734118964) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 14:43:10 (1734118990) mds.MDS.mdt.nrs_policies=tbf oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg204-client: oleg204-server: ssh exited with exit code 36 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg204-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg204-client: oleg204-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 14:43:19 (1734118999) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 9537 kill md_thread xargs PASS 77r (39s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 14:43:58 (1734119038) oleg204-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg204-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg204-client: oleg204-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 14:44:00 (1734119040) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506590 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:44:00.000000000 -0500 Modify: 2024-12-13 14:44:00.000000000 -0500 Change: 2024-12-13 14:44:00.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 14:44:12 (1734119052) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 14:44:13 (1734119053) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (0s) == sanityn test 81a: rename and stat under striped directory ========================================================== 14:44:13 (1734119053) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 14:44:14 (1734119054) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 14:44:15 (1734119055) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 14:44:16 (1734119056) total: 2500 open/close in 4.72 seconds: 529.96 ops/second snapshot_time 1734119074.052040485 secs.nsecs start_time 1734119056.969899261 secs.nsecs elapsed_time 17.082141224 secs.nsecs open 2505 samples [usecs] 56 1451 683345 197711243 close 2504 samples [usecs] 51 465 191911 15507447 mknod 2501 samples [usecs] 156 1421 610775 159202827 unlink 2500 samples [usecs] 273 8605 1063177 941605585 mkdir 2 samples [usecs] 403 649 1052 583610 rmdir 2 samples [usecs] 1966 3375 5341 15255781 rename 2500 samples [usecs] 231 2283 2197078 2656961586 getattr 3760 samples [usecs] 5 97 53720 924642 parallel_rename_file 2500 samples [usecs] 231 2283 2197078 2656961586 crossdir_rename 2500 samples [usecs] 231 2283 2197078 2656961586 PASS 81d (19s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 14:44:35 (1734119075) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6987 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6987 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 14:44:36 (1734119076) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 14:44:37 (1734119077) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 85: Lustre API root cache race =========== 14:44:44 (1734119084) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1734119084 Finishing test test1 at 1734119085 PASS 85 (2s) == sanityn test 90: open/create and unlink striped directory ========================================================== 14:44:46 (1734119086) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 14:44:47 (1734119087) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 14:44:48 (1734119088) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (0s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 14:44:48 (1734119088) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5911 0x1717 0x240000400 1 5896 0x1708 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5897 0x1709 0x280000400 0 5912 0x1718 0x240000400 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 14:44:56 (1734119096) osc.lustre-OST0000-osc-ffff8800b54d5800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b6d7d000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b54d5800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b6d7d000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0248447 s, 330 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00031 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5743: 29872 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0115884 s, 353 kB/s osc.lustre-OST0000-osc-ffff8800b54d5800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6d7d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b54d5800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6d7d000.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 14:45:06 (1734119106) llite.lustre-ffff8800b54d5800.fast_read=0 llite.lustre-ffff8800b6d7d000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0137681 s, 152 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800b54d5800.fast_read=1 llite.lustre-ffff8800b6d7d000.fast_read=1 PASS 95a (19s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 14:45:25 (1734119125) llite.lustre-ffff8800b54d5800.fast_read=0 llite.lustre-ffff8800b6d7d000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00178361 s, 6.9 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b54d5800.fast_read=1 llite.lustre-ffff8800b6d7d000.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 14:45:32 (1734119132) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 14:45:33 (1734119133) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00146506 s, 2.8 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 14:45:35 (1734119135) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0193374 s, 108 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 14:45:36 (1734119136) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0225671 s, 92.9 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 14:45:38 (1734119138) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.001759 s, 6.8 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 14:45:40 (1734119140) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000678901 s, 6.0 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 14:45:41 (1734119141) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000497368 s, 8.2 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 14:45:43 (1734119143) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000529247 s, 7.7 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 14:45:46 (1734119146) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1734119147 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1734119149 Finishing test test23 at 1734119149 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 14:45:53 (1734119153) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (0s) == sanityn test 105: Glimpse and lock cancel race ======== 14:45:53 (1734119153) fail_loc=0x416 PASS 105 (27s) == sanityn test 106a: Verify the btime via statx() ======= 14:46:20 (1734119180) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 14:46:21 (1734119181) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 14:46:22 (1734119182) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 14:46:23 (1734119183) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0553299 s, 190 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5919 0x171f 0x240000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 First grouplock blocks second one PASS 107a (3s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 14:46:26 (1734119186) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0580807 s, 181 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5903 0x170f 0x280000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6987 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 14:46:34 (1734119194) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00159978 s, 6.3 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.16759 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 14:46:40 (1734119200) 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 10 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 15 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 16 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 18 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 19 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 22 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 23 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 28 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/pcc_cpufreq/refcnt': 'No such device' oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 34 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 35 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 36 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 192.168.202.104@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.104@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg204-client.virtnet /mnt/lustre (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg204-server' oleg204-server: oleg204-server.virtnet: executing load_modules_local oleg204-server: Loading modules from /home/green/git/lustre-release/lustre oleg204-server: detected 4 online CPUs by sysfs oleg204-server: MODOPTS_LIBCFS= oleg204-server: Force libcfs to create 2 CPU partitions oleg204-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre Starting client: oleg204-client.virtnet: -o user_xattr,flock 192.168.202.104@tcp:/lustre /mnt/lustre2 PASS 109 (395s) == sanityn test 110: do not grant another lock on resend ========================================================== 14:53:15 (1734119595) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-12-13 14:53:15.000000000 -0500 Modify: 2024-12-13 14:53:15.000000000 -0500 Change: 2024-12-13 14:53:15.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg204-client.virtnet oleg204-client.virtnet PASS 110 (166s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 14:56:01 (1734119761) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 14:56:02 (1734119762) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (0s) == sanityn test 113: check servers of specified fs ======= 14:56:02 (1734119762) lustre-OST0000-osc-ffff8800925fa800 active. lustre-OST0001-osc-ffff8800925fa800 active. lustre-OST0000-osc-ffff8800925fa800 active. lustre-OST0001-osc-ffff8800925fa800 active. MGC192.168.202.104@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 14:56:04 (1734119764) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 14:56:05 (1734119765) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (0s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 14:56:05 (1734119765) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 14:56:06 (1734119766) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=22 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1141 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1143 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.1224 s, 0.2 kB/s , 20.1177 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1207 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1165 s, 0.2 kB/s , 20.1193 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1245 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1218 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.0889 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2038 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2217 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.2204 s, 0.1 kB/s , 40.208 s, 0.1 kB/s , 40.2063 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.2236 s, 0.1 kB/s , 40.2022 s, 0.1 kB/s , 40.2186 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2223 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2295 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3151 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3124 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 60.3345 s, 0.1 kB/s , 60.3114 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 60.2895 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 60.329 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 60.2909 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3123 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.3365 s, 0.1 kB/s , 80.3379 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3656 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3922 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4054 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.4065 s, 0.1 kB/s , 80.4421 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4051 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 100.451 s, 0.0 kB/s , 100.432 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 100.436 s, 0.0 kB/s 4096 bytes (4.1 kB) copied, 100.442 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.45 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.522 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.529 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.532 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800925fa800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800925fa800.ost_server_uuid in FULL state after 0 sec PASS 200 (105s) cleanup: ====================================================== == sanityn test complete, duration 4550 sec ============== 14:57:52 (1734119872) === sanityn: start cleanup 14:57:52 (1734119872) === Stopping clients: oleg204-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg204-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 14:58:26 (1734119906) ===