-----============= acceptance-small: sanityn ============----- Fri Feb 14 06:41:17 EST 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg435-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 06:41:28 (1739533288) === Starting client oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Started clients oleg435-client.virtnet: 192.168.204.135@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg435-client.virtnet: executing check_config_client /mnt/lustre oleg435-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg435-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800ad422000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012dd4c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800ad422000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012dd4c000.idle_timeout=debug disable quota as required oleg435-server: oleg435-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 06:41:43 (1739533303) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217262 s, 48.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7021] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 06:41:44 (1739533304) PASS 0 (4s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 06:41:48 (1739533308) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (4s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 06:41:52 (1739533312) -rw-r--r-- 1 root root 0 Feb 14 06:41 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 06:41:56 (1739533316) -rw-r--r-- 1 root root 0 Feb 14 06:41 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 06:41:59 (1739533319) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 06:42:03 (1739533323) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 06:42:07 (1739533327) total 1 -rwxrwxrwx 1 root root 0 Feb 14 06:41 f2a -rwxrwxrwx 1 root root 0 Feb 14 06:41 f2b total 1 -rwxrwxrwx 1 root root 0 Feb 14 06:41 f2a -rwxrwxrwx 1 root root 0 Feb 14 06:41 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.0187474 s, 27.3 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 06:42:11 (1739533331) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 06:42:13 (1739533333) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.351212 s, 6.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 06:42:17 (1739533337) PASS 3 (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 06:42:21 (1739533341) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 06:42:26 (1739533346) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 06:42:30 (1739533350) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 06:42:33 (1739533353) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (4s) == sanityn test 8: remove of open special file on other node ==================================================================== 06:42:37 (1739533357) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 06:42:41 (1739533361) PASS 9a (5s) == sanityn test 9b: append to striped sparse file ======== 06:42:46 (1739533366) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314788 s, 33.3 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0547947 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (5s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 06:42:51 (1739533371) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0191392 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00756587 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0172122 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.014038 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0212703 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00594242 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00927397 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00793887 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00862894 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0135619 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0100379 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0206684 s, 0.0 kB/s PASS 10a (5s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 06:42:56 (1739533376) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.016971 s, 181 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0304872 s, 134 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000830051 s, 3.7 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 06:43:00 (1739533380) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 06:43:04 (1739533384) warning: '-runtime' deprecated, use '-t runtime' instead - link 554 (time 1739533395.79 total 10.01 last 55.36) - link 1055 (time 1739533405.79 total 20.01 last 50.09) - link 1543 (time 1739533415.81 total 30.03 last 48.71) - link 2091 (time 1739533425.81 total 40.03 last 54.79) - link 2549 (time 1739533435.82 total 50.04 last 45.75) - link 3138 (time 1739533445.83 total 60.05 last 58.84) - link 3847 (time 1739533455.83 total 70.05 last 70.89) - link 4439 (time 1739533465.84 total 80.06 last 59.16) - link 4988 (time 1739533475.84 total 90.06 last 54.88) total: 5548 link in 100.00 seconds: 55.48 ops/second using seed 339182652 running for 100 seconds - stat 10000 (time 1739533394 ; total 8 ; last 8) - stat 20000 (time 1739533402 ; total 16 ; last 8) - stat 30000 (time 1739533409 ; total 23 ; last 7) - stat 40000 (time 1739533417 ; total 31 ; last 8) - stat 50000 (time 1739533425 ; total 39 ; last 8) - stat 60000 (time 1739533433 ; total 47 ; last 8) - stat 66420 (time 1739533444 ; total 58 ; last 11) - stat 69832 (time 1739533455 ; total 69 ; last 11) - stat 70000 (time 1739533455 ; total 69 ; last 0) - stat 72863 (time 1739533466 ; total 80 ; last 11) - stat 76109 (time 1739533477 ; total 91 ; last 11) total: 78282 stats in 100 seconds: 782.820007 stats/second - unlinked 0 (time 1739533436 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile5548) error: No such file or directory total: 5547 unlinks in 86 seconds: 64.500000 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22082) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22085) - No such process PASS 12 (141s) == sanityn test 13: test directory page revocation ======= 06:45:25 (1739533525) f13 f13 PASS 13 (6s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 06:45:31 (1739533531) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4866: 23480 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 06:45:35 (1739533535) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4866: 24118 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (5s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 06:45:40 (1739533540) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24753 Terminated $DIR1/$tdir/sleep 60 PASS 14b (4s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 06:45:44 (1739533544) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25391 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 06:45:48 (1739533548) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 26027 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 06:45:52 (1739533552) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (20s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 06:46:12 (1739533572) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.459863 s, 18.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.382605 s, 21.9 MB/s Chance of close/open is 1 in 50 Seed set to 3575 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x86049b 000003[0] 1739533575.989551 trunc from 00000000 to 0x86049a (0x86049b bytes) 000005[1] 1739533576.063113 trunc from 0x86049b to 0x5b21b8 (0x2ae2e2 bytes) 000007[0] 1739533576.149540 trunc from 0x5b21b9 to 0x0fc4aa (0x4b5d0e bytes) 000009[0] 1739533576.183928 write 0x412b19 thru 0x419045 (0x652d bytes) 000011[1] 1739533576.213656 mapread 0x052856 thru 0x052f18 (0x6c3 bytes) 000014[1] 1739533576.267942 trunc from 0x419046 to 0x4ef70f (0xd66ca bytes) 000015[0] 1739533576.319010 read 0x4bae84 thru 0x4c145d (0x65da bytes) 000016[1] 1739533576.373473 read 0x103238 thru 0x10ea6e (0xb837 bytes) 000018[0] 1739533576.409756 trunc from 0x4ef710 to 0x21245c (0x2dd2b3 bytes) 000020[0] 1739533576.476467 write 0x0a37af thru 0x0a5675 (0x1ec7 bytes) 000022[0] 1739533576.505346 write 0x122a27 thru 0x12e979 (0xbf53 bytes) 000026[1] 1739533576.520645 mapwrite 0x6a4ceb thru 0x6b413e (0xf454 bytes) 000031[1] 1739533576.739321 read 0x157fbf thru 0x15ffae (0x7ff0 bytes) 000033[0] 1739533576.767968 read 0x1783af thru 0x17f6bb (0x730d bytes) 000034[0] 1739533576.787014 write 0x2a19dd thru 0x2a8785 (0x6da9 bytes) 000035[0] 1739533576.819714 mapwrite 0x24d098 thru 0x24f2e1 (0x224a bytes) 000037[0] 1739533576.951490 write 0x48a097 thru 0x4927ac (0x8716 bytes) 000039[0] 1739533577.005840 mapwrite 0x87ba62 thru 0x889cc6 (0xe265 bytes) 000040[0] 1739533577.118122 mapread 0x14f4ab thru 0x1525a9 (0x30ff bytes) 000042[0] 1739533577.167960 write 0x6666de thru 0x675a54 (0xf377 bytes) 000049[1] 1739533577.226681 write 0x82beb2 thru 0x830627 (0x4776 bytes) 000050[1] 1739533577.267162 trunc from 0x889cc7 to 0x2329d0 (0x6572f6 bytes) 000051[1] 1739533577.343880 trunc from 0x2329d1 to 0x2a868a (0x75cba bytes) 000052[1] 1739533577.369449 mapread 0x2865e0 thru 0x293155 (0xcb76 bytes) 000056[0] 1739533577.433742 mapread 0x26f83a thru 0x27f639 (0xfe00 bytes) 000058[1] 1739533577.506521 mapread 0x14161c thru 0x14165f (0x44 bytes) 000059[0] 1739533577.511770 write 0x831b51 thru 0x8410f3 (0xf5a3 bytes) 000060[1] 1739533577.541722 mapwrite 0x459baf thru 0x45cbbe (0x3010 bytes) truncating to largest ever: 0x9edbbb 000064[1] 1739533577.821371 trunc from 0x8410f4 to 0x9edbba (0x1acac7 bytes) 000069[1] 1739533577.877322 mapwrite 0x7b9b0d thru 0x7beca8 (0x519c bytes) 000070[0] 1739533577.957153 mapread 0x9775ab thru 0x97e5fe (0x7054 bytes) 000071[1] 1739533577.994145 trunc from 0x9edbbb to 0x2eccbf (0x700efb bytes) 000072[1] 1739533578.027216 mapread 0x0c199a thru 0x0d11e7 (0xf84e bytes) 000074[0] 1739533578.083885 write 0x250434 thru 0x2567d7 (0x63a4 bytes) 000076[0] 1739533578.124486 read 0x2a5a4f thru 0x2a9842 (0x3df4 bytes) 000082[1] 1739533578.292912 mapread 0x0a8200 thru 0x0af0e9 (0x6eea bytes) 000084[1] 1739533578.398524 read 0x0944cd thru 0x09655e (0x2092 bytes) 000085[1] 1739533578.501112 trunc from 0x2eccc0 to 0x15e5c3 (0x18e6fc bytes) 000086[1] 1739533578.592404 write 0x540011 thru 0x5406fa (0x6ea bytes) 000087[0] 1739533578.598699 read 0x1eeeda thru 0x1f3595 (0x46bc bytes) 000090[1] 1739533578.687837 write 0x1c2910 thru 0x1cba57 (0x9148 bytes) 000093[0] 1739533578.727955 write 0x2743c6 thru 0x27b233 (0x6e6e bytes) 000094[1] 1739533578.760945 read 0x274855 thru 0x27bc24 (0x73d0 bytes) 000096[1] 1739533578.804434 read 0x1b27d3 thru 0x1bb55a (0x8d88 bytes) 000099[1] 1739533578.830965 read 0x4f40c0 thru 0x4fb983 (0x78c4 bytes) 000102[1] 1739533578.838143 read 0x50f189 thru 0x518867 (0x96df bytes) 000106[1] 1739533578.854619 trunc from 0x5406fb to 0x8457c6 (0x3050cc bytes) 000111[1] 1739533578.942174 mapwrite 0x22a334 thru 0x231cae (0x797b bytes) 000113[0] 1739533579.051900 write 0x427949 thru 0x42a3e8 (0x2aa0 bytes) 000114[0] 1739533579.113104 mapwrite 0x1e0256 thru 0x1e7ad3 (0x787e bytes) 000115[1] 1739533579.218701 read 0x448629 thru 0x44929f (0xc77 bytes) 000117[1] 1739533579.278521 read 0x381989 thru 0x381bb5 (0x22d bytes) 000119[1] 1739533579.302616 read 0x2934b8 thru 0x29762c (0x4175 bytes) 000120[1] 1739533579.309188 read 0x5ffdb8 thru 0x60bef4 (0xc13d bytes) 000124[0] 1739533579.317780 read 0x70946d thru 0x70d046 (0x3bda bytes) 000125[0] 1739533579.326151 read 0x4b55bb thru 0x4b8829 (0x326f bytes) 000128[0] 1739533579.331259 read 0x3cf8ae thru 0x3dc591 (0xcce4 bytes) 000129[1] 1739533579.339362 read 0x423cfa thru 0x42503a (0x1341 bytes) 000132[0] 1739533579.344024 write 0x9a0f87 thru 0x9a571c (0x4796 bytes) 000136[0] 1739533579.383125 trunc from 0x9a571d to 0x7effc5 (0x1b5757 bytes) 000137[1] 1739533579.449659 mapread 0x630402 thru 0x63203d (0x1c3c bytes) 000138[0] 1739533579.506707 read 0x08b138 thru 0x0926d3 (0x759c bytes) 000139[0] 1739533579.523459 trunc from 0x7effc6 to 0x4d7d11 (0x3182b4 bytes) 000143[0] 1739533579.578867 mapread 0x01cd6e thru 0x02763c (0xa8cf bytes) 000144[0] 1739533579.601075 mapread 0x23b0a2 thru 0x249c77 (0xebd6 bytes) 000146[1] 1739533579.624368 mapwrite 0x1377f7 thru 0x14343f (0xbc49 bytes) 000147[0] 1739533579.752756 mapwrite 0x4faf6e thru 0x4ffd28 (0x4dbb bytes) 000148[0] 1739533579.866079 trunc from 0x4ffd29 to 0x6489d9 (0x148cb1 bytes) 000150[0] 1739533579.882567 write 0x85f893 thru 0x86a7a1 (0xaf0f bytes) 000152[0] 1739533579.908336 write 0x286fe7 thru 0x289720 (0x273a bytes) 000153[0] 1739533579.942657 write 0x0b2dd8 thru 0x0b3bc5 (0xdee bytes) 000154[0] 1739533579.955691 write 0x45492f thru 0x45586c (0xf3e bytes) 000155[1] 1739533579.967963 write 0x8a2879 thru 0x8b1220 (0xe9a8 bytes) 000156[0] 1739533580.018786 mapwrite 0x7be243 thru 0x7c3d58 (0x5b16 bytes) 000157[1] 1739533580.284859 trunc from 0x8b1221 to 0x7f82d4 (0xb8f4c bytes) 000158[0] 1739533580.354029 mapwrite 0x43b28a thru 0x446a43 (0xb7ba bytes) 000160[0] 1739533580.492388 mapwrite 0x953fba thru 0x95f165 (0xb1ac bytes) 000161[1] 1739533580.633842 mapread 0x2da4e1 thru 0x2e7e8e (0xd9ae bytes) 000162[1] 1739533580.705965 read 0x1e6ec8 thru 0x1f1566 (0xa69f bytes) 000163[1] 1739533580.760318 mapread 0x31eb4e thru 0x32a835 (0xbce8 bytes) 000165[0] 1739533580.824687 read 0x4a1ee1 thru 0x4a9253 (0x7373 bytes) 000168[0] 1739533580.880176 mapwrite 0x456b0a thru 0x45fa7d (0x8f74 bytes) 000176[0] 1739533580.988139 trunc from 0x95f166 to 0x66220f (0x2fcf56 bytes) 000177[1] 1739533581.026550 write 0x41ca12 thru 0x41ecd1 (0x22c0 bytes) 000178[1] 1739533581.070804 write 0x342718 thru 0x34c616 (0x9eff bytes) 000179[1] 1739533581.094781 trunc from 0x662210 to 0x9eda83 (0x38b874 bytes) 000182[0] 1739533581.145613 mapread 0x5b9604 thru 0x5c4731 (0xb12e bytes) 000185[1] 1739533581.212380 write 0x8957de thru 0x898b24 (0x3347 bytes) 000186[0] 1739533581.230361 mapread 0x711840 thru 0x7140d7 (0x2898 bytes) 000188[0] 1739533581.251570 write 0x45ecc0 thru 0x46238d (0x36ce bytes) 000189[0] 1739533581.279835 mapwrite 0x516929 thru 0x523cd0 (0xd3a8 bytes) 000191[1] 1739533581.370638 mapread 0x0a9093 thru 0x0aead9 (0x5a47 bytes) 000193[1] 1739533581.411244 mapread 0x550be5 thru 0x551602 (0xa1e bytes) 000194[0] 1739533581.446717 mapwrite 0x3282a6 thru 0x32e539 (0x6294 bytes) 000195[0] 1739533581.584442 read 0x27f8fd thru 0x28d189 (0xd88d bytes) 000196[1] 1739533581.601674 trunc from 0x9eda84 to 0x4b579b (0x5382e8 bytes) 000198[0] 1739533581.677231 mapwrite 0x848825 thru 0x84e8da (0x60b6 bytes) 000199[0] 1739533581.776491 read 0x3a8a26 thru 0x3b455a (0xbb35 bytes) 000201[0] 1739533581.804735 mapread 0x2f30cc thru 0x2f3c90 (0xbc5 bytes) 000202[0] 1739533581.811384 mapread 0x42c419 thru 0x43c284 (0xfe6c bytes) 000203[1] 1739533581.829204 read 0x019a65 thru 0x024e6d (0xb409 bytes) 000205[0] 1739533581.870190 trunc from 0x84e8db to 0x6aa99a (0x1a3f40 bytes) 000207[1] 1739533581.921140 mapread 0x000852 thru 0x0075ca (0x6d79 bytes) 000210[0] 1739533581.972602 read 0x36bf90 thru 0x370d6d (0x4dde bytes) 000212[1] 1739533582.010977 read 0x2c7f91 thru 0x2d6d78 (0xede8 bytes) 000213[0] 1739533582.047552 read 0x09f4fe thru 0x0a257d (0x3080 bytes) 000214[0] 1739533582.086322 trunc from 0x6aa99b to 0x79a1a9 (0xef80f bytes) 000215[1] 1739533582.120714 read 0x53ddfe thru 0x54216d (0x4370 bytes) 000217[0] 1739533582.169025 write 0x4d44b7 thru 0x4d77ca (0x3314 bytes) 000219[0] 1739533582.203143 write 0x946425 thru 0x94c9ab (0x6587 bytes) 000222[1] 1739533582.207063 trunc from 0x94c9ac to 0x0b91e1 (0x8937ca bytes) 000224[0] 1739533582.303611 mapwrite 0x4fc78c thru 0x4fd0c2 (0x937 bytes) 000227[1] 1739533582.442325 trunc from 0x4fd0c3 to 0x3b7789 (0x145939 bytes) 000237[1] 1739533582.627977 mapread 0x369f1c thru 0x36de97 (0x3f7c bytes) 000238[0] 1739533582.665636 mapwrite 0x246f15 thru 0x24be50 (0x4f3c bytes) 000241[0] 1739533582.809715 mapwrite 0x641c9a thru 0x64c8ac (0xac13 bytes) 000242[0] 1739533582.996717 read 0x4a4a08 thru 0x4a6928 (0x1f21 bytes) 000256[1] 1739533583.170506 mapwrite 0x4bcccd thru 0x4c6764 (0x9a98 bytes) 000260[1] 1739533583.299763 read 0x2b5a53 thru 0x2b6ce7 (0x1295 bytes) 000261[1] 1739533583.309781 mapread 0x635983 thru 0x63b463 (0x5ae1 bytes) 000262[0] 1739533583.343774 trunc from 0x64c8ad to 0x4e9216 (0x163696 bytes) 000264[1] 1739533583.438874 read 0x3d3a6b thru 0x3e07f6 (0xcd8c bytes) 000265[0] 1739533583.472191 mapwrite 0x18b05c thru 0x18b31c (0x2c1 bytes) 000266[0] 1739533583.560147 mapread 0x2fa5e9 thru 0x2fbb9e (0x15b6 bytes) 000267[0] 1739533583.594341 read 0x2aad72 thru 0x2b4987 (0x9c16 bytes) 000268[1] 1739533583.605481 trunc from 0x4e9217 to 0x7160b4 (0x22ce9e bytes) 000271[0] 1739533583.690164 mapread 0x10d99c thru 0x1146f6 (0x6d5b bytes) 000273[1] 1739533583.769069 mapwrite 0x49f1ed thru 0x4a2ee4 (0x3cf8 bytes) 000274[0] 1739533583.865364 trunc from 0x7160b5 to 0x0d1208 (0x644eac bytes) 000275[1] 1739533583.907025 write 0x20919d thru 0x20a022 (0xe86 bytes) 000277[0] 1739533583.964989 mapwrite 0x36177d thru 0x363dce (0x2652 bytes) 000280[0] 1739533584.162895 write 0x34fd33 thru 0x355646 (0x5914 bytes) 000282[0] 1739533584.194511 write 0x115e48 thru 0x1221e1 (0xc39a bytes) 000285[1] 1739533584.247505 mapread 0x08977a thru 0x08e9a0 (0x5227 bytes) 000288[1] 1739533584.340310 trunc from 0x363dcf to 0x5cb0e0 (0x267312 bytes) 000291[0] 1739533584.385690 mapread 0x572451 thru 0x57e978 (0xc528 bytes) 000293[0] 1739533584.459933 write 0x4f10da thru 0x4f40fd (0x3024 bytes) 000294[1] 1739533584.492477 write 0x42e996 thru 0x4392fc (0xa967 bytes) 000296[0] 1739533584.527841 mapwrite 0x46a14c thru 0x470e8a (0x6d3f bytes) 000297[0] 1739533584.643094 mapwrite 0x29ccdd thru 0x2a014d (0x3471 bytes) 000298[1] 1739533584.725777 read 0x15b474 thru 0x16a56d (0xf0fa bytes) 000299[1] 1739533584.763246 read 0x1a80ab thru 0x1b26eb (0xa641 bytes) 000300[1] 1739533584.787185 read 0x458264 thru 0x461706 (0x94a3 bytes) 000305[0] 1739533584.848825 mapwrite 0x589537 thru 0x58e10f (0x4bd9 bytes) 000307[0] 1739533584.975637 mapwrite 0x1d4c84 thru 0x1e1ba1 (0xcf1e bytes) 000309[0] 1739533585.127967 mapwrite 0x81b761 thru 0x81d642 (0x1ee2 bytes) 000310[1] 1739533585.215816 read 0x249555 thru 0x24d425 (0x3ed1 bytes) 000311[1] 1739533585.246982 mapread 0x33eecf thru 0x3436ef (0x4821 bytes) 000312[1] 1739533585.264100 write 0x201402 thru 0x2056af (0x42ae bytes) 000313[0] 1739533585.298980 write 0x47d258 thru 0x481785 (0x452e bytes) 000315[1] 1739533585.350146 mapread 0x3e5307 thru 0x3e664d (0x1347 bytes) 000319[1] 1739533585.478621 mapwrite 0x147f54 thru 0x149e1b (0x1ec8 bytes) 000320[1] 1739533585.564459 trunc from 0x81d643 to 0x19199a (0x68bca8 bytes) 000321[1] 1739533585.602037 mapwrite 0x40e1b6 thru 0x414550 (0x639b bytes) 000322[1] 1739533585.680010 trunc from 0x414551 to 0x808e46 (0x3f48f6 bytes) 000323[1] 1739533585.697159 mapwrite 0x3c5a69 thru 0x3c9aa6 (0x403e bytes) 000324[1] 1739533585.786898 read 0x6751db thru 0x67c739 (0x755f bytes) 000327[0] 1739533585.817834 read 0x13bf6f thru 0x143445 (0x74d7 bytes) 000332[1] 1739533585.894531 trunc from 0x808e47 to 0x050e38 (0x7b800e bytes) 000334[1] 1739533585.945369 write 0x1ef668 thru 0x1fe758 (0xf0f1 bytes) 000335[0] 1739533585.969137 mapwrite 0x10bd1b thru 0x118b4e (0xce34 bytes) 000336[1] 1739533586.087115 write 0x37e582 thru 0x37f94f (0x13ce bytes) 000337[1] 1739533586.094266 trunc from 0x37f950 to 0x589872 (0x209f23 bytes) 000343[0] 1739533586.138756 mapwrite 0x3e6c3f thru 0x3e8e5b (0x221d bytes) 000351[0] 1739533586.347176 write 0x37c31f thru 0x38bcc3 (0xf9a5 bytes) 000352[1] 1739533586.372455 trunc from 0x589873 to 0x1759c3 (0x413eaf bytes) 000359[1] 1739533586.462148 trunc from 0x1759c4 to 0x762daf (0x5ed3ec bytes) 000361[1] 1739533586.510560 mapread 0x0075e8 thru 0x00b19c (0x3bb5 bytes) 000364[0] 1739533586.557697 mapwrite 0x4ab456 thru 0x4b49ec (0x9597 bytes) 000365[1] 1739533586.682352 mapread 0x2510d4 thru 0x25594a (0x4877 bytes) 000366[0] 1739533586.715272 mapread 0x33cb8f thru 0x345769 (0x8bdb bytes) 000367[1] 1739533586.768849 read 0x418a4e thru 0x422d2b (0xa2de bytes) 000368[1] 1739533586.811617 write 0x80fd70 thru 0x814d34 (0x4fc5 bytes) 000372[0] 1739533586.858833 read 0x6336c5 thru 0x640e15 (0xd751 bytes) 000374[1] 1739533586.896618 mapread 0x7c0403 thru 0x7c0a2b (0x629 bytes) 000375[1] 1739533586.903304 mapwrite 0x49eba8 thru 0x4a1fdd (0x3436 bytes) 000378[1] 1739533587.022910 mapread 0x769b79 thru 0x76a249 (0x6d1 bytes) 000381[1] 1739533587.044552 read 0x3383fb thru 0x339b36 (0x173c bytes) 000383[1] 1739533587.068013 trunc from 0x814d35 to 0x80e568 (0x67cc bytes) 000384[1] 1739533587.104649 read 0x319bd0 thru 0x3209cc (0x6dfd bytes) 000385[1] 1739533587.113123 trunc from 0x80e569 to 0x9399e0 (0x12b478 bytes) 000389[1] 1739533587.188750 read 0x67d69c thru 0x68a917 (0xd27c bytes) 000390[0] 1739533587.195340 mapwrite 0x754088 thru 0x757507 (0x3480 bytes) 000393[1] 1739533587.313973 mapread 0x3dc45d thru 0x3eb6ae (0xf252 bytes) 000395[1] 1739533587.402747 read 0x21c72a thru 0x21f958 (0x322f bytes) 000397[1] 1739533587.451305 trunc from 0x9399e1 to 0x5d8d7a (0x360c66 bytes) 000401[1] 1739533587.538387 mapread 0x30f20d thru 0x30f9b1 (0x7a5 bytes) 000402[1] 1739533587.546455 trunc from 0x5d8d7b to 0x75d3fc (0x184682 bytes) 000405[0] 1739533587.573350 mapread 0x63e7f8 thru 0x64c4bb (0xdcc4 bytes) 000406[1] 1739533587.632116 write 0x8e3d2e thru 0x8f2c95 (0xef68 bytes) 000410[1] 1739533587.668174 read 0x79f34f thru 0x7a1486 (0x2138 bytes) 000413[1] 1739533587.698133 trunc from 0x8f2c96 to 0x325f27 (0x5ccd6e bytes) 000414[0] 1739533587.748250 mapwrite 0x71b8ef thru 0x7239d3 (0x80e5 bytes) 000417[1] 1739533587.905300 mapread 0x44451a thru 0x44fa05 (0xb4ec bytes) 000420[0] 1739533587.969605 read 0x07dda1 thru 0x088e69 (0xb0c9 bytes) 000422[1] 1739533587.994417 read 0x29b608 thru 0x29ed06 (0x36ff bytes) 000424[1] 1739533588.002686 write 0x39f263 thru 0x3ab344 (0xc0e2 bytes) 000425[0] 1739533588.038621 read 0x20e63a thru 0x21a96d (0xc334 bytes) 000426[1] 1739533588.080760 read 0x0abca7 thru 0x0b5e48 (0xa1a2 bytes) 000428[0] 1739533588.086199 write 0x4d986c thru 0x4dd08f (0x3824 bytes) 000429[1] 1739533588.117251 write 0x132c47 thru 0x13fdf9 (0xd1b3 bytes) 000430[0] 1739533588.145876 mapread 0x2edc80 thru 0x2f0670 (0x29f1 bytes) 000432[0] 1739533588.198187 trunc from 0x7239d4 to 0x6a5869 (0x7e16a bytes) 000433[1] 1739533588.229493 write 0x5fdcc9 thru 0x60393a (0x5c72 bytes) 000434[1] 1739533588.257154 mapread 0x38103a thru 0x38935e (0x8325 bytes) 000437[0] 1739533588.325488 write 0x940e53 thru 0x94b07b (0xa229 bytes) 000438[0] 1739533588.360178 mapwrite 0x47a51b thru 0x47b1a7 (0xc8d bytes) 000439[0] 1739533588.441661 write 0x07a621 thru 0x0874d6 (0xceb6 bytes) 000441[0] 1739533588.470944 read 0x5d16b4 thru 0x5d2e35 (0x1782 bytes) 000442[1] 1739533588.474585 trunc from 0x94b07c to 0x122f07 (0x828174 bytes) 000445[0] 1739533588.525363 mapwrite 0x4f36b3 thru 0x4fae57 (0x77a5 bytes) 000447[1] 1739533588.657430 write 0x47cdb3 thru 0x48a267 (0xd4b5 bytes) 000452[1] 1739533588.768794 write 0x208ae1 thru 0x20a98e (0x1eae bytes) 000454[0] 1739533588.789401 trunc from 0x4fae58 to 0x4f0537 (0xa920 bytes) 000455[0] 1739533588.842250 read 0x27eb9b thru 0x27febe (0x1324 bytes) 000456[0] 1739533588.849911 write 0x31bc74 thru 0x32430c (0x8699 bytes) 000458[1] 1739533588.909473 read 0x42959c thru 0x42cd73 (0x37d8 bytes) 000459[1] 1739533588.938212 write 0x56e37e thru 0x576413 (0x8096 bytes) 000460[1] 1739533588.988154 trunc from 0x576414 to 0x2d05bf (0x2a5e54 bytes) 000464[1] 1739533589.067102 write 0x7a1663 thru 0x7af0d4 (0xda72 bytes) 000465[0] 1739533589.087989 read 0x02a223 thru 0x02cc5e (0x2a3c bytes) 000466[1] 1739533589.108294 read 0x2800ab thru 0x284fa7 (0x4efd bytes) 000471[0] 1739533589.149906 mapread 0x5c3f70 thru 0x5cc8ee (0x897f bytes) 000473[1] 1739533589.186805 trunc from 0x7af0d5 to 0x64ac1a (0x1644ba bytes) 000475[1] 1739533589.235496 read 0x55dc5f thru 0x5627fc (0x4b9e bytes) 000479[0] 1739533589.254422 read 0x1d3f6e thru 0x1d9dba (0x5e4d bytes) 000486[1] 1739533589.378823 mapwrite 0x3dfee2 thru 0x3e0d52 (0xe71 bytes) 000488[0] 1739533589.489582 trunc from 0x64ac1b to 0x8c71b8 (0x27c59e bytes) 000490[1] 1739533589.544883 read 0x551e3e thru 0x554c4d (0x2e10 bytes) 000491[0] 1739533589.572674 trunc from 0x8c71b9 to 0x8d20bd (0xaf05 bytes) 000492[0] 1739533589.612912 read 0x4675f0 thru 0x467b83 (0x594 bytes) 000494[1] 1739533589.647389 read 0x6df58b thru 0x6e258f (0x3005 bytes) 000495[1] 1739533589.680821 mapwrite 0x3507cc thru 0x35e4c7 (0xdcfc bytes) 000498[1] 1739533589.802610 mapwrite 0x5c64de thru 0x5d5966 (0xf489 bytes) 000499[0] 1739533589.928719 mapread 0x3bb55a thru 0x3caef9 (0xf9a0 bytes) 000500[0] 1739533589.982376 read 0x1c8829 thru 0x1d876e (0xff46 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 3590 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1739533590.508780 write_OD 0x270000 thru 0x27afff (0xb000 bytes) truncating to largest ever: 0x73fdec 000002[1] 1739533590.593321 trunc from 0x27b000 to 0x73fdeb (0x4c4dec bytes) 000008[0] 1739533590.656672 write 0x53d000 thru 0x544781 (0x7782 bytes) 000009[0] 1739533590.693582 trunc from 0x73fdec to 0x376d72 (0x3c9079 bytes) 000013[1] 1739533590.808845 mapwrite 0x0ca000 thru 0x0d906a (0xf06b bytes) 000015[0] 1739533591.001393 mapwrite 0x44c000 thru 0x45b87d (0xf87e bytes) 000016[0] 1739533591.115593 write 0x178000 thru 0x178c08 (0xc09 bytes) 000023[0] 1739533591.173996 mapread 0x098000 thru 0x099efe (0x1eff bytes) 000024[0] 1739533591.213661 mapread 0x009000 thru 0x00dd9a (0x4d9b bytes) 000025[1] 1739533591.263793 read_OD 0x268000 thru 0x26efff (0x7000 bytes) 000026[1] 1739533591.294755 trunc from 0x45b87e to 0x4790d2 (0x1d855 bytes) 000028[0] 1739533591.341769 mapwrite 0x274000 thru 0x27edcf (0xadd0 bytes) 000033[0] 1739533591.450766 write 0x274000 thru 0x27c302 (0x8303 bytes) 000036[0] 1739533591.463415 trunc from 0x4790d3 to 0x4888df (0xf80d bytes) 000037[1] 1739533591.500991 mapwrite 0x137000 thru 0x13b4ed (0x44ee bytes) 000038[1] 1739533591.576666 mapwrite 0x6b0000 thru 0x6b1c31 (0x1c32 bytes) 000039[1] 1739533591.650584 mapwrite 0x952000 thru 0x954706 (0x2707 bytes) 000040[0] 1739533591.722826 trunc from 0x954707 to 0x07a4a4 (0x8da262 bytes) 000041[1] 1739533591.754915 mapwrite 0x9e7000 thru 0x9efa8f (0x8a90 bytes) 000042[1] 1739533591.852776 mapread 0x697000 thru 0x6a27fc (0xb7fd bytes) 000044[0] 1739533591.902561 mapread 0x133000 thru 0x141765 (0xe766 bytes) 000046[1] 1739533591.943968 mapread 0x695000 thru 0x699125 (0x4126 bytes) truncating to largest ever: 0x74be00 000048[0] 1739533591.959367 trunc from 0x9efa90 to 0x74bdff (0x2a3c90 bytes) 000049[1] 1739533592.021995 read_OD 0x1de000 thru 0x1ebfff (0xe000 bytes) 000050[1] 1739533592.056247 read_OD 0x744000 thru 0x74afff (0x7000 bytes) 000052[0] 1739533592.065159 read 0x6e5000 thru 0x6f4556 (0xf557 bytes) 000053[1] 1739533592.079165 mapwrite 0x2c5000 thru 0x2cd6b3 (0x86b4 bytes) 000054[0] 1739533592.154902 mapwrite 0x0b8000 thru 0x0c42f3 (0xc2f4 bytes) 000056[1] 1739533592.254516 mapread 0x6b9000 thru 0x6c69b1 (0xd9b2 bytes) 000059[0] 1739533592.287758 read 0x53d000 thru 0x53ef73 (0x1f74 bytes) 000061[1] 1739533592.292380 mapwrite 0x644000 thru 0x64e942 (0xa943 bytes) 000062[0] 1739533592.379224 read 0x49a000 thru 0x4a8b0c (0xeb0d bytes) 000064[1] 1739533592.395230 mapread 0x301000 thru 0x30bfbb (0xafbc bytes) 000065[0] 1739533592.424269 write 0x14b000 thru 0x1598d0 (0xe8d1 bytes) 000066[1] 1739533592.460945 read_OD 0x6c3000 thru 0x6c3fff (0x1000 bytes) 000071[0] 1739533592.491654 mapwrite 0x47f000 thru 0x48a4ec (0xb4ed bytes) 000072[1] 1739533592.571481 mapwrite 0x624000 thru 0x624bc2 (0xbc3 bytes) 000073[0] 1739533592.635288 trunc from 0x74be00 to 0x2b3d30 (0x4980cf bytes) 000074[0] 1739533592.683605 mapread 0x021000 thru 0x029cdd (0x8cde bytes) 000075[1] 1739533592.701881 mapwrite 0x425000 thru 0x431670 (0xc671 bytes) 000078[1] 1739533592.802672 mapread 0x27f000 thru 0x28a174 (0xb175 bytes) 000079[0] 1739533592.827991 mapread 0x351000 thru 0x35d512 (0xc513 bytes) 000080[0] 1739533592.869959 write 0x4b8000 thru 0x4c6be8 (0xebe9 bytes) 000081[0] 1739533592.881230 mapwrite 0x0d7000 thru 0x0dc38d (0x538e bytes) 000082[1] 1739533592.957857 read_OD 0x38c000 thru 0x391fff (0x6000 bytes) 000084[0] 1739533592.986596 mapwrite 0x96d000 thru 0x974e45 (0x7e46 bytes) 000087[0] 1739533593.085194 read 0x807000 thru 0x8088ba (0x18bb bytes) 000088[1] 1739533593.100684 read_OD 0x7dd000 thru 0x7e3fff (0x7000 bytes) 000093[1] 1739533593.123631 mapread 0x485000 thru 0x488abf (0x3ac0 bytes) 000094[0] 1739533593.135628 read 0x43f000 thru 0x4419b7 (0x29b8 bytes) 000097[1] 1739533593.139780 mapwrite 0x44c000 thru 0x450e8c (0x4e8d bytes) 000100[1] 1739533593.223751 trunc from 0x974e46 to 0x551580 (0x4238c5 bytes) 000102[0] 1739533593.238915 trunc from 0x551581 to 0x085797 (0x4cbde9 bytes) 000105[1] 1739533593.290255 mapwrite 0x307000 thru 0x307642 (0x643 bytes) 000106[1] 1739533593.408553 trunc from 0x307643 to 0x1a6eef (0x160753 bytes) 000108[0] 1739533593.443623 write 0x5fb000 thru 0x609683 (0xe684 bytes) 000110[0] 1739533593.457108 trunc from 0x609684 to 0x34d090 (0x2bc5f3 bytes) 000111[0] 1739533593.501525 read 0x0fb000 thru 0x1018b1 (0x68b2 bytes) 000113[0] 1739533593.598099 write 0x99e000 thru 0x9acf72 (0xef73 bytes) truncating to largest ever: 0x7b20aa 000114[1] 1739533593.619017 trunc from 0x9acf73 to 0x7b20a9 (0x1faec9 bytes) 000115[1] 1739533593.658614 read_OD 0x529000 thru 0x52afff (0x2000 bytes) 000117[0] 1739533593.672268 write 0x01f000 thru 0x022209 (0x320a bytes) 000118[0] 1739533593.708611 mapwrite 0x656000 thru 0x660092 (0xa093 bytes) 000120[1] 1739533593.814320 trunc from 0x7b20aa to 0x22844e (0x589c5b bytes) 000122[1] 1739533593.877131 mapread 0x03a000 thru 0x049555 (0xf556 bytes) 000123[1] 1739533593.907619 mapread 0x0fe000 thru 0x106fb0 (0x8fb1 bytes) truncating to largest ever: 0x8894e6 000126[0] 1739533593.944402 trunc from 0x22844f to 0x8894e5 (0x661097 bytes) 000127[1] 1739533593.986509 mapwrite 0x686000 thru 0x6882fc (0x22fd bytes) 000129[0] 1739533594.089915 mapwrite 0x62c000 thru 0x62eb2f (0x2b30 bytes) 000133[0] 1739533594.198394 mapwrite 0x7c4000 thru 0x7ce8a1 (0xa8a2 bytes) 000138[1] 1739533594.335250 mapwrite 0x302000 thru 0x30ba61 (0x9a62 bytes) 000140[1] 1739533594.443504 mapwrite 0x0b6000 thru 0x0bd36d (0x736e bytes) 000142[0] 1739533594.501583 mapread 0x434000 thru 0x434ffd (0xffe bytes) 000149[1] 1739533594.518621 mapwrite 0x68f000 thru 0x69a387 (0xb388 bytes) 000150[0] 1739533594.602224 read 0x034000 thru 0x037624 (0x3625 bytes) 000151[0] 1739533594.618238 trunc from 0x8894e6 to 0x65739b (0x23214a bytes) 000152[1] 1739533594.647713 write_OD 0x6d0000 thru 0x6d7fff (0x8000 bytes) 000153[1] 1739533594.744593 trunc from 0x6d8000 to 0x03a16b (0x69de94 bytes) 000154[1] 1739533594.799907 read_OD 0x023000 thru 0x02ffff (0xd000 bytes) 000157[0] 1739533594.817149 trunc from 0x03a16c to 0x046924 (0xc7b9 bytes) 000159[1] 1739533594.840931 read_OD 0x043000 thru 0x045fff (0x3000 bytes) 000160[0] 1739533594.862608 read 0x038000 thru 0x043e13 (0xbe14 bytes) 000171[0] 1739533594.892485 read 0x02d000 thru 0x039d8a (0xcd8b bytes) 000174[1] 1739533594.893267 read_OD 0x00c000 thru 0x00ffff (0x4000 bytes) 000179[0] 1739533594.898829 trunc from 0x046925 to 0x5ec8c3 (0x5a5f9f bytes) 000181[1] 1739533594.941987 read_OD 0x306000 thru 0x307fff (0x2000 bytes) 000182[0] 1739533594.952229 mapread 0x5a2000 thru 0x5afd6d (0xdd6e bytes) 000185[0] 1739533595.051214 read 0x06f000 thru 0x07b2d5 (0xc2d6 bytes) 000189[0] 1739533595.053277 read 0x0a4000 thru 0x0a5db4 (0x1db5 bytes) 000191[0] 1739533595.053648 write 0x76d000 thru 0x77441d (0x741e bytes) 000197[0] 1739533595.122297 trunc from 0x77441e to 0x859bca (0xe57ad bytes) 000199[0] 1739533595.149340 mapread 0x4fe000 thru 0x507a8d (0x9a8e bytes) 000202[1] 1739533595.228016 trunc from 0x859bcb to 0x62fa6a (0x22a160 bytes) 000204[1] 1739533595.290130 mapwrite 0x21c000 thru 0x229107 (0xd108 bytes) 000205[0] 1739533595.423097 mapwrite 0x73c000 thru 0x749f5d (0xdf5e bytes) 000208[0] 1739533595.555276 mapread 0x28c000 thru 0x29a9b5 (0xe9b6 bytes) 000211[1] 1739533595.629014 trunc from 0x749f5e to 0x87b3b0 (0x131453 bytes) 000213[0] 1739533595.656740 write 0x9c5000 thru 0x9ce014 (0x9015 bytes) 000215[0] 1739533595.668770 mapread 0x903000 thru 0x90f1ce (0xc1cf bytes) 000216[1] 1739533595.689534 write_OD 0x485000 thru 0x48dfff (0x9000 bytes) 000218[0] 1739533595.748645 write 0x550000 thru 0x558f4a (0x8f4b bytes) 000219[1] 1739533595.765748 read_OD 0x71a000 thru 0x71ffff (0x6000 bytes) 000220[1] 1739533595.806581 write_OD 0x5ea000 thru 0x5f3fff (0xa000 bytes) 000221[0] 1739533595.871017 read 0x132000 thru 0x13a7fd (0x87fe bytes) 000223[1] 1739533595.884945 write_OD 0x7b8000 thru 0x7b9fff (0x2000 bytes) 000225[0] 1739533595.930115 write_OD 0x1f6000 thru 0x1fcfff (0x7000 bytes) truncating to largest ever: 0x98ccc8 000226[0] 1739533595.969483 trunc from 0x9ce015 to 0x98ccc7 (0x4134d bytes) 000228[1] 1739533596.005262 mapread 0x632000 thru 0x64066f (0xe670 bytes) 000230[1] 1739533596.048674 write_OD 0x8db000 thru 0x8e2fff (0x8000 bytes) 000235[0] 1739533596.099510 trunc from 0x98ccc8 to 0x57b864 (0x411463 bytes) 000240[1] 1739533596.138091 mapread 0x005000 thru 0x00d2ce (0x82cf bytes) 000241[0] 1739533596.170297 mapwrite 0x3a3000 thru 0x3a9f2f (0x6f30 bytes) 000245[0] 1739533596.277333 read_OD 0x1f3000 thru 0x200fff (0xe000 bytes) 000246[1] 1739533596.292580 trunc from 0x57b865 to 0x10caf3 (0x46ed71 bytes) 000247[1] 1739533596.320666 read_OD 0x002000 thru 0x007fff (0x6000 bytes) 000248[1] 1739533596.332284 read_OD 0x0b7000 thru 0x0c4fff (0xe000 bytes) 000249[1] 1739533596.344314 trunc from 0x10caf4 to 0x3355ec (0x228af9 bytes) 000251[1] 1739533596.397687 mapwrite 0x706000 thru 0x70fe99 (0x9e9a bytes) 000252[1] 1739533596.575390 read_OD 0x69a000 thru 0x6a6fff (0xd000 bytes) 000254[1] 1739533596.605886 trunc from 0x70fe9a to 0x5bff52 (0x14ff47 bytes) 000259[1] 1739533596.676161 trunc from 0x5bff53 to 0x21d270 (0x3a2ce2 bytes) 000265[1] 1739533596.713207 mapwrite 0x488000 thru 0x4923cf (0xa3d0 bytes) 000269[1] 1739533596.854040 trunc from 0x4923d0 to 0x32339c (0x16f033 bytes) 000270[1] 1739533596.873824 mapread 0x213000 thru 0x21e311 (0xb312 bytes) 000272[1] 1739533596.893134 write_OD 0x596000 thru 0x598fff (0x3000 bytes) 000274[1] 1739533596.956675 trunc from 0x599000 to 0x48c8f3 (0x10c70c bytes) 000277[0] 1739533596.995717 mapread 0x018000 thru 0x02757f (0xf580 bytes) 000278[1] 1739533597.029962 trunc from 0x48c8f4 to 0x618461 (0x18bb6e bytes) 000280[1] 1739533597.053462 read_OD 0x4d1000 thru 0x4d1fff (0x1000 bytes) 000282[0] 1739533597.070665 mapread 0x51d000 thru 0x5213c6 (0x43c7 bytes) 000284[1] 1739533597.082649 trunc from 0x618462 to 0x69ebe4 (0x86783 bytes) 000285[1] 1739533597.132069 mapwrite 0x081000 thru 0x0889fa (0x79fb bytes) 000286[1] 1739533597.217687 write_OD 0x55e000 thru 0x56afff (0xd000 bytes) 000288[1] 1739533597.283360 trunc from 0x69ebe5 to 0x4f5b72 (0x1a9072 bytes) 000291[0] 1739533597.323723 mapread 0x29b000 thru 0x2a0259 (0x525a bytes) 000292[1] 1739533597.356664 mapread 0x37b000 thru 0x37d829 (0x282a bytes) 000293[1] 1739533597.371964 write_OD 0x95b000 thru 0x969fff (0xf000 bytes) 000294[1] 1739533597.428964 mapread 0x30b000 thru 0x317275 (0xc276 bytes) 000296[1] 1739533597.460368 mapwrite 0x907000 thru 0x914450 (0xd451 bytes) 000297[1] 1739533597.563918 trunc from 0x96a000 to 0x497c91 (0x4d236e bytes) 000299[0] 1739533597.603542 mapread 0x0c1000 thru 0x0c2c48 (0x1c49 bytes) 000300[1] 1739533597.648585 mapwrite 0x9a6000 thru 0x9ac1a4 (0x61a5 bytes) 000303[1] 1739533597.805821 mapread 0x27a000 thru 0x281b33 (0x7b34 bytes) 000304[1] 1739533597.839037 read_OD 0x0d5000 thru 0x0d7fff (0x3000 bytes) 000308[1] 1739533597.861811 trunc from 0x9ac1a5 to 0x38a8c8 (0x6218dc bytes) 000310[0] 1739533597.888458 mapread 0x014000 thru 0x01e131 (0xa132 bytes) 000313[0] 1739533597.951481 mapread 0x295000 thru 0x2a34ab (0xe4ac bytes) 000314[0] 1739533597.999823 write_OD 0x39e000 thru 0x3a8fff (0xb000 bytes) 000315[1] 1739533598.089649 read_OD 0x2c8000 thru 0x2c8fff (0x1000 bytes) 000316[0] 1739533598.097782 mapwrite 0x093000 thru 0x095bd6 (0x2bd7 bytes) truncating to largest ever: 0x98fe3c 000317[0] 1739533598.191010 trunc from 0x3a9000 to 0x98fe3b (0x5e6e3c bytes) 000319[0] 1739533598.214272 mapread 0x14b000 thru 0x155da9 (0xadaa bytes) 000320[1] 1739533598.239210 mapread 0x808000 thru 0x8147de (0xc7df bytes) 000321[1] 1739533598.372445 write_OD 0x131000 thru 0x138fff (0x8000 bytes) 000322[0] 1739533598.445816 read_OD 0x372000 thru 0x37efff (0xd000 bytes) 000325[1] 1739533598.459249 trunc from 0x98fe3c to 0x5dc9ac (0x3b348f bytes) 000330[1] 1739533598.528190 write_OD 0x94e000 thru 0x953fff (0x6000 bytes) 000332[0] 1739533598.581193 trunc from 0x954000 to 0x4cd826 (0x4867d9 bytes) 000333[0] 1739533598.616583 mapwrite 0x342000 thru 0x34666f (0x4670 bytes) 000334[1] 1739533598.702019 trunc from 0x4cd827 to 0x1f1a92 (0x2dbd94 bytes) 000335[0] 1739533598.733693 write_OD 0x6ce000 thru 0x6d9fff (0xc000 bytes) 000338[0] 1739533598.811426 mapread 0x201000 thru 0x206df9 (0x5dfa bytes) 000342[0] 1739533598.833031 read_OD 0x697000 thru 0x697fff (0x1000 bytes) 000344[0] 1739533598.839143 write_OD 0x8a9000 thru 0x8b3fff (0xb000 bytes) 000345[0] 1739533598.912948 write_OD 0x7ae000 thru 0x7b7fff (0xa000 bytes) 000346[1] 1739533598.987660 trunc from 0x8b4000 to 0x4a5b3e (0x40e4c1 bytes) 000348[1] 1739533599.062270 mapwrite 0x604000 thru 0x60adb4 (0x6db5 bytes) 000350[0] 1739533599.197837 write_OD 0x3d1000 thru 0x3d7fff (0x7000 bytes) 000351[1] 1739533599.257722 mapread 0x276000 thru 0x277547 (0x1548 bytes) 000353[0] 1739533599.268707 mapread 0x602000 thru 0x608236 (0x6237 bytes) 000354[0] 1739533599.295252 trunc from 0x60adb5 to 0x4df43c (0x12b978 bytes) 000356[0] 1739533599.345948 trunc from 0x4df43d to 0x190d9a (0x34e6a2 bytes) 000359[1] 1739533599.399719 trunc from 0x190d9b to 0x618b56 (0x487dbc bytes) 000361[1] 1739533599.458901 mapread 0x501000 thru 0x506432 (0x5433 bytes) 000363[0] 1739533599.500528 mapread 0x5b9000 thru 0x5bd81e (0x481f bytes) 000367[1] 1739533599.531227 read_OD 0x478000 thru 0x478fff (0x1000 bytes) 000368[1] 1739533599.536984 read_OD 0x079000 thru 0x084fff (0xc000 bytes) 000369[0] 1739533599.546757 mapread 0x27c000 thru 0x27e198 (0x2199 bytes) 000370[1] 1739533599.574354 trunc from 0x618b57 to 0x377bd4 (0x2a0f82 bytes) 000372[0] 1739533599.628801 read_OD 0x0af000 thru 0x0b4fff (0x6000 bytes) 000375[1] 1739533599.666850 write_OD 0x749000 thru 0x756fff (0xe000 bytes) 000377[1] 1739533599.764028 read_OD 0x3d1000 thru 0x3d7fff (0x7000 bytes) 000379[1] 1739533599.776716 trunc from 0x757000 to 0x7c7f54 (0x70f55 bytes) 000380[1] 1739533599.830344 mapwrite 0x2e5000 thru 0x2eda66 (0x8a67 bytes) 000383[1] 1739533600.008722 read_OD 0x2cf000 thru 0x2d1fff (0x3000 bytes) 000389[0] 1739533600.047339 read_OD 0x522000 thru 0x524fff (0x3000 bytes) 000392[1] 1739533600.061663 mapwrite 0x32a000 thru 0x33566d (0xb66e bytes) 000397[0] 1739533600.191348 read_OD 0x333000 thru 0x337fff (0x5000 bytes) 000399[0] 1739533600.223089 read_OD 0x41c000 thru 0x424fff (0x9000 bytes) 000400[0] 1739533600.235903 read_OD 0x3f1000 thru 0x3fcfff (0xc000 bytes) 000401[0] 1739533600.244378 read_OD 0x5d4000 thru 0x5ddfff (0xa000 bytes) 000402[0] 1739533600.252906 mapwrite 0x36d000 thru 0x371181 (0x4182 bytes) 000403[1] 1739533600.341876 mapread 0x2b7000 thru 0x2b82f8 (0x12f9 bytes) 000404[0] 1739533600.373611 read_OD 0x672000 thru 0x675fff (0x4000 bytes) 000408[1] 1739533600.386246 write_OD 0x0f4000 thru 0x0fdfff (0xa000 bytes) 000409[0] 1739533600.453528 mapwrite 0x176000 thru 0x17f953 (0x9954 bytes) 000412[0] 1739533600.551011 mapwrite 0x6a2000 thru 0x6a8c42 (0x6c43 bytes) 000419[1] 1739533600.706078 mapread 0x4f7000 thru 0x4fec99 (0x7c9a bytes) 000421[0] 1739533600.749766 mapwrite 0x1eb000 thru 0x1ef6ec (0x46ed bytes) 000422[1] 1739533600.819304 mapwrite 0x42c000 thru 0x43a680 (0xe681 bytes) 000423[1] 1739533600.931917 read_OD 0x42e000 thru 0x436fff (0x9000 bytes) 000431[0] 1739533600.960697 read_OD 0x4d5000 thru 0x4d8fff (0x4000 bytes) 000433[1] 1739533600.978842 trunc from 0x7c7f55 to 0x2cdab7 (0x4fa49d bytes) 000434[1] 1739533601.033937 read_OD 0x16a000 thru 0x16bfff (0x2000 bytes) 000439[1] 1739533601.053464 write_OD 0x499000 thru 0x4a0fff (0x8000 bytes) 000440[0] 1739533601.155399 mapread 0x16d000 thru 0x17179b (0x479c bytes) 000442[0] 1739533601.186729 mapwrite 0x602000 thru 0x6086ff (0x6700 bytes) 000444[0] 1739533601.355306 trunc from 0x608700 to 0x97baf2 (0x3733f3 bytes) 000445[1] 1739533601.381474 write_OD 0x1ae000 thru 0x1bafff (0xd000 bytes) 000446[1] 1739533601.458602 mapwrite 0x755000 thru 0x75df2b (0x8f2c bytes) 000447[0] 1739533601.701110 write_OD 0x44e000 thru 0x450fff (0x3000 bytes) 000448[0] 1739533601.818159 read_OD 0x10b000 thru 0x10efff (0x4000 bytes) 000449[0] 1739533601.830351 mapread 0x897000 thru 0x89eadf (0x7ae0 bytes) 000450[0] 1739533601.857390 mapwrite 0x967000 thru 0x974e10 (0xde11 bytes) 000451[1] 1739533602.005099 trunc from 0x97baf3 to 0x270603 (0x70b4ef bytes) 000454[0] 1739533602.073350 trunc from 0x270604 to 0x5504b4 (0x2dfeb1 bytes) 000459[0] 1739533602.143316 trunc from 0x5504b5 to 0x90f928 (0x3bf474 bytes) 000461[1] 1739533602.194663 write_OD 0x9fd000 thru 0x9fefff (0x2000 bytes) 000464[1] 1739533602.249603 trunc from 0x9ff000 to 0x31bcfe (0x6e3301 bytes) 000467[0] 1739533602.294781 write_OD 0x4ef000 thru 0x4f7fff (0x9000 bytes) 000469[1] 1739533602.367976 mapread 0x2db000 thru 0x2e50f6 (0xa0f7 bytes) 000470[0] 1739533602.445268 mapread 0x1c5000 thru 0x1cbd44 (0x6d45 bytes) 000478[1] 1739533602.490250 trunc from 0x4f8000 to 0x3ff2ca (0xf8d35 bytes) 000484[1] 1739533602.571943 mapwrite 0x76c000 thru 0x76c281 (0x282 bytes) 000485[1] 1739533602.669957 mapwrite 0x4e2000 thru 0x4f091d (0xe91e bytes) truncating to largest ever: 0x9d9e9c 000486[0] 1739533602.856134 trunc from 0x76c282 to 0x9d9e9b (0x26dc1a bytes) 000489[1] 1739533602.902889 trunc from 0x9d9e9c to 0x109e30 (0x8d006b bytes) 000491[0] 1739533602.976912 mapread 0x012000 thru 0x017947 (0x5948 bytes) 000493[1] 1739533603.019455 trunc from 0x109e31 to 0x0354f2 (0xd493e bytes) 000494[1] 1739533603.044782 read_OD 0x01f000 thru 0x022fff (0x4000 bytes) 000495[1] 1739533603.066445 write_OD 0x767000 thru 0x76dfff (0x7000 bytes) 000496[0] 1739533603.135534 trunc from 0x76e000 to 0x38d9a6 (0x3e0659 bytes) 000497[1] 1739533603.178710 trunc from 0x38d9a7 to 0x71111e (0x383778 bytes) 000498[0] 1739533603.197969 read_OD 0x497000 thru 0x4a0fff (0xa000 bytes) 000500[1] 1739533603.218851 mapwrite 0x4f0000 thru 0x4fcc23 (0xcc24 bytes) All operations completed A-OK! PASS 16a (33s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 06:46:45 (1739533605) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.348517 s, 24.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.277832 s, 30.2 MB/s Chance of close/open is 1 in 50 Seed set to 3607 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1ea172 000002[0] 1739533608.064159 trunc from 00000000 to 0x1ea171 (0x1ea172 bytes) 000004[1] 1739533608.115726 write 0x35cd35 thru 0x35dba6 (0xe72 bytes) 000005[0] 1739533608.133022 write 0x3c0654 thru 0x3c189d (0x124a bytes) 000006[0] 1739533608.149915 mapread 0x34a0e8 thru 0x34b39a (0x12b3 bytes) 000007[0] 1739533608.177202 read 0x2ddf54 thru 0x2de94b (0x9f8 bytes) 000008[1] 1739533608.184703 write 0x266cd2 thru 0x26738e (0x6bd bytes) 000010[0] 1739533608.234018 trunc from 0x3c189e to 0x01cd49 (0x3a4b54 bytes) 000016[0] 1739533608.366158 read 0x00780c thru 0x009358 (0x1b4d bytes) 000017[1] 1739533608.397336 mapwrite 0x051ace thru 0x052aa5 (0xfd8 bytes) 000020[1] 1739533608.613366 mapread 0x005785 thru 0x006e53 (0x16cf bytes) 000021[1] 1739533608.641953 write 0x82c487 thru 0x82de08 (0x1982 bytes) 000022[1] 1739533608.656524 write 0x258a25 thru 0x258dd7 (0x3b3 bytes) 000024[0] 1739533608.677539 mapread 0x41e321 thru 0x41fe9d (0x1b7d bytes) truncating to largest ever: 0x5c432c 000025[0] 1739533608.721061 trunc from 0x82de09 to 0x5c432b (0x269add bytes) 000026[0] 1739533608.782906 write 0x51734a thru 0x517941 (0x5f8 bytes) 000030[1] 1739533608.833280 mapread 0x10ca50 thru 0x10df4c (0x14fd bytes) truncating to largest ever: 0x79ccd2 000031[1] 1739533608.886877 trunc from 0x5c432c to 0x79ccd1 (0x1d89a6 bytes) 000032[1] 1739533608.918934 write 0x449b90 thru 0x44acf6 (0x1167 bytes) 000034[1] 1739533608.955419 write 0x3329b4 thru 0x332a57 (0xa4 bytes) 000035[0] 1739533608.982870 read 0x75ce55 thru 0x75e4d8 (0x1684 bytes) 000040[1] 1739533609.055470 read 0x508278 thru 0x509ee1 (0x1c6a bytes) 000041[1] 1739533609.061665 trunc from 0x79ccd2 to 0x34fe25 (0x44ceac bytes) 000044[0] 1739533609.112406 trunc from 0x34fe26 to 0x2dc8a1 (0x73584 bytes) 000046[0] 1739533609.155568 read 0x0c5ca8 thru 0x0c6244 (0x59d bytes) truncating to largest ever: 0x81e959 000049[1] 1739533609.238541 trunc from 0x2dc8a2 to 0x81e958 (0x5420b7 bytes) 000050[0] 1739533609.299941 mapread 0x3a759c thru 0x3a9215 (0x1c7a bytes) 000051[0] 1739533609.359103 mapread 0x2939b5 thru 0x293df7 (0x443 bytes) 000052[0] 1739533609.375611 trunc from 0x81e959 to 0x0989a2 (0x785fb6 bytes) 000053[0] 1739533609.410545 read 0x0602e1 thru 0x060486 (0x1a6 bytes) 000055[0] 1739533609.460042 read 0x08f139 thru 0x090c47 (0x1b0f bytes) 000056[0] 1739533609.474712 write 0x2903a4 thru 0x2913c0 (0x101d bytes) 000058[0] 1739533609.486090 mapwrite 0x1c3a71 thru 0x1c4cb2 (0x1242 bytes) truncating to largest ever: 0x991379 000062[0] 1739533609.573394 trunc from 0x2913c1 to 0x991378 (0x6fffb8 bytes) 000063[1] 1739533609.614081 mapwrite 0x1c7511 thru 0x1c8dd6 (0x18c6 bytes) 000064[1] 1739533609.715617 mapread 0x8067ca thru 0x8075d4 (0xe0b bytes) 000065[1] 1739533609.741075 write 0x6935b1 thru 0x694dc9 (0x1819 bytes) 000066[0] 1739533609.768093 mapread 0x247706 thru 0x24932c (0x1c27 bytes) 000067[0] 1739533609.819828 write 0x104bf2 thru 0x105ef9 (0x1308 bytes) 000069[0] 1739533609.873429 write 0x355364 thru 0x35660a (0x12a7 bytes) 000070[1] 1739533609.892845 trunc from 0x991379 to 0x11eb4b (0x87282d bytes) 000071[1] 1739533609.920801 mapread 0x0c444c thru 0x0c48db (0x490 bytes) 000072[1] 1739533609.924936 write 0x26c5f6 thru 0x26e140 (0x1b4b bytes) 000074[0] 1739533609.948031 mapread 0x16cb79 thru 0x16cbe1 (0x69 bytes) 000076[0] 1739533609.979427 mapwrite 0x4b14ac thru 0x4b21ca (0xd1f bytes) 000078[1] 1739533610.161537 mapread 0x04ba74 thru 0x04d40f (0x199c bytes) 000080[1] 1739533610.196083 trunc from 0x4b21cb to 0x75b276 (0x2a90ac bytes) 000085[1] 1739533610.296263 trunc from 0x75b277 to 0x7e5620 (0x8a3aa bytes) 000086[0] 1739533610.319489 trunc from 0x7e5621 to 0x92ddf9 (0x1487d9 bytes) 000089[0] 1739533610.414929 mapread 0x08a694 thru 0x08afd7 (0x944 bytes) 000095[0] 1739533610.478326 read 0x18c733 thru 0x18e5d5 (0x1ea3 bytes) 000099[0] 1739533610.520949 write 0x1f500b thru 0x1f5f54 (0xf4a bytes) 000101[0] 1739533610.569112 write 0x1f2ca7 thru 0x1f48c6 (0x1c20 bytes) 000106[1] 1739533610.629136 mapwrite 0x452f93 thru 0x4542bd (0x132b bytes) 000107[1] 1739533610.747759 read 0x0d78bf thru 0x0d8e90 (0x15d2 bytes) 000108[0] 1739533610.790944 mapread 0x10fb72 thru 0x1104e5 (0x974 bytes) 000109[1] 1739533610.813928 mapwrite 0x8759a9 thru 0x875c52 (0x2aa bytes) 000112[1] 1739533610.915317 trunc from 0x92ddfa to 0x59eb88 (0x38f271 bytes) 000113[1] 1739533610.936927 trunc from 0x59eb89 to 0x40bd36 (0x192e52 bytes) 000114[1] 1739533610.975497 mapwrite 0x232878 thru 0x2337de (0xf67 bytes) 000118[0] 1739533611.102262 mapwrite 0x2dc2c6 thru 0x2de0cc (0x1e07 bytes) 000119[1] 1739533611.224148 write 0x3191f0 thru 0x3195be (0x3cf bytes) 000123[1] 1739533611.312664 read 0x1ca9f2 thru 0x1cbf2d (0x153c bytes) 000124[1] 1739533611.330758 mapread 0x3c54ff thru 0x3c5cf2 (0x7f4 bytes) 000126[1] 1739533611.335779 write 0x666716 thru 0x6676b3 (0xf9e bytes) 000127[1] 1739533611.337722 mapwrite 0x2d3453 thru 0x2d346e (0x1c bytes) 000129[1] 1739533611.416201 trunc from 0x6676b4 to 0x6b7ec7 (0x50814 bytes) 000132[0] 1739533611.469611 mapread 0x48ec43 thru 0x48ff5a (0x1318 bytes) 000134[0] 1739533611.515683 mapwrite 0x5ecd19 thru 0x5eddac (0x1094 bytes) 000135[0] 1739533611.602145 write 0x063023 thru 0x0643d0 (0x13ae bytes) 000136[1] 1739533611.625943 mapread 0x082452 thru 0x0841bb (0x1d6a bytes) 000137[0] 1739533611.679271 trunc from 0x6b7ec8 to 0x795533 (0xdd66c bytes) 000139[1] 1739533611.718035 trunc from 0x795534 to 0x11b42d (0x67a106 bytes) 000140[1] 1739533611.756515 trunc from 0x11b42e to 0x9755e4 (0x85a1b7 bytes) 000144[0] 1739533611.815720 mapread 0x3578b0 thru 0x357da7 (0x4f8 bytes) 000147[0] 1739533611.864322 mapwrite 0x889b5a thru 0x88b996 (0x1e3d bytes) 000148[1] 1739533611.964307 trunc from 0x9755e5 to 0x617546 (0x35e09e bytes) 000149[1] 1739533612.000957 trunc from 0x617547 to 0x1d1a19 (0x445b2d bytes) 000151[1] 1739533612.049183 mapread 0x05fc15 thru 0x061172 (0x155e bytes) 000153[1] 1739533612.120042 mapwrite 0x10bc6c thru 0x10db67 (0x1efc bytes) 000154[1] 1739533612.188355 trunc from 0x1d1a1a to 0x20f72d (0x3dd14 bytes) 000156[0] 1739533612.237582 mapread 0x0b772b thru 0x0b8735 (0x100b bytes) 000157[0] 1739533612.289952 read 0x1aa8eb thru 0x1abab1 (0x11c7 bytes) 000159[1] 1739533612.363981 mapwrite 0x5c92b8 thru 0x5ca133 (0xe7c bytes) 000160[1] 1739533612.483723 read 0x0f0dbf thru 0x0f1dff (0x1041 bytes) 000169[0] 1739533612.573420 write 0x1e850e thru 0x1e9f17 (0x1a0a bytes) 000170[0] 1739533612.624190 read 0x571038 thru 0x5727c4 (0x178d bytes) 000171[1] 1739533612.647887 mapread 0x4ccae7 thru 0x4ce635 (0x1b4f bytes) 000172[1] 1739533612.663143 mapread 0x5a7635 thru 0x5a824c (0xc18 bytes) 000174[0] 1739533612.706750 read 0x4f0bcc thru 0x4f1c9a (0x10cf bytes) 000176[1] 1739533612.724766 trunc from 0x5ca134 to 0x02d02e (0x59d105 bytes) 000178[0] 1739533612.763127 read 0x001704 thru 0x001ef5 (0x7f2 bytes) 000179[1] 1739533612.779912 write 0x99244b thru 0x9927eb (0x3a1 bytes) 000183[0] 1739533612.801745 mapwrite 0x735c59 thru 0x7368c9 (0xc71 bytes) 000184[0] 1739533612.948566 write 0x1ac3a9 thru 0x1ae26e (0x1ec6 bytes) truncating to largest ever: 0x9ad4ae 000185[1] 1739533612.999159 trunc from 0x9927ec to 0x9ad4ad (0x1acc2 bytes) 000186[0] 1739533613.036958 mapread 0x53094a thru 0x5315bc (0xc73 bytes) 000187[1] 1739533613.080611 trunc from 0x9ad4ae to 0x315282 (0x69822b bytes) 000188[1] 1739533613.104098 read 0x263b91 thru 0x2640df (0x54f bytes) 000191[0] 1739533613.166041 trunc from 0x315283 to 0x85f3ab (0x54a129 bytes) 000193[0] 1739533613.218470 mapread 0x074708 thru 0x075723 (0x101c bytes) 000197[1] 1739533613.256869 read 0x5d5f4c thru 0x5d7d7d (0x1e32 bytes) 000198[0] 1739533613.299141 trunc from 0x85f3ac to 0x2045ed (0x65adbe bytes) 000199[1] 1739533613.329840 write 0x70f12e thru 0x710b27 (0x19fa bytes) 000200[1] 1739533613.362025 read 0x11a49a thru 0x11b61e (0x1185 bytes) 000203[0] 1739533613.399397 trunc from 0x710b28 to 0x356ca5 (0x3b9e82 bytes) 000204[1] 1739533613.442473 write 0x883431 thru 0x884f34 (0x1b04 bytes) 000205[0] 1739533613.473799 trunc from 0x884f35 to 0x9aba73 (0x126b3f bytes) 000206[0] 1739533613.533689 read 0x142fdc thru 0x143205 (0x22a bytes) 000207[0] 1739533613.549380 mapread 0x8e8d03 thru 0x8e9ccd (0xfcb bytes) 000210[0] 1739533613.571687 mapwrite 0x5c8324 thru 0x5c982b (0x1508 bytes) 000211[1] 1739533613.626668 mapwrite 0x3aec23 thru 0x3b0a67 (0x1e45 bytes) 000213[0] 1739533613.760514 trunc from 0x9aba74 to 0x86cb9f (0x13eed4 bytes) 000214[1] 1739533613.804310 mapwrite 0x557051 thru 0x558bb3 (0x1b63 bytes) 000218[1] 1739533613.914108 mapwrite 0x8eb363 thru 0x8ec6ce (0x136c bytes) 000222[0] 1739533614.029440 write 0x4d4ffc thru 0x4d65df (0x15e4 bytes) 000224[1] 1739533614.085683 trunc from 0x8ec6cf to 0x13691b (0x7b5db3 bytes) 000225[1] 1739533614.133919 write 0x0a5c22 thru 0x0a5d01 (0xe0 bytes) 000226[0] 1739533614.151836 write 0x4d2fbe thru 0x4d4a33 (0x1a76 bytes) 000227[0] 1739533614.172792 mapread 0x21d607 thru 0x21f0cd (0x1ac7 bytes) 000228[0] 1739533614.220911 write 0x9fb7a5 thru 0x9fcd74 (0x15d0 bytes) 000230[1] 1739533614.244092 mapwrite 0x8dd8a3 thru 0x8dd956 (0xb4 bytes) 000232[0] 1739533614.377756 mapwrite 0x7cf6a4 thru 0x7d006d (0x9ca bytes) 000233[0] 1739533614.489945 read 0x5480c1 thru 0x548aa5 (0x9e5 bytes) 000234[1] 1739533614.506073 mapread 0x0091c5 thru 0x009729 (0x565 bytes) 000235[0] 1739533614.534011 write 0x23f442 thru 0x240745 (0x1304 bytes) 000238[1] 1739533614.563216 mapread 0x5cf147 thru 0x5d0a41 (0x18fb bytes) 000242[0] 1739533614.591165 mapwrite 0x5f7866 thru 0x5f85af (0xd4a bytes) 000249[1] 1739533614.675752 trunc from 0x9fcd75 to 0x857e93 (0x1a4ee1 bytes) 000253[1] 1739533614.707970 mapwrite 0x11a4c0 thru 0x11afb6 (0xaf7 bytes) 000255[0] 1739533614.793216 mapread 0x09933e thru 0x09ae78 (0x1b3b bytes) 000257[1] 1739533614.848105 read 0x3ead8d thru 0x3ecd1f (0x1f93 bytes) truncating to largest ever: 0x9fa6ad 000259[1] 1739533614.871698 trunc from 0x857e94 to 0x9fa6ac (0x1a2819 bytes) 000260[0] 1739533614.917196 write 0x6d49e5 thru 0x6d5b99 (0x11b5 bytes) 000262[1] 1739533614.982593 read 0x8c7e8f thru 0x8c9287 (0x13f9 bytes) 000270[1] 1739533615.115193 mapwrite 0x51a1cb thru 0x51a762 (0x598 bytes) 000271[1] 1739533615.195930 mapwrite 0x9e6f8e thru 0x9e819d (0x1210 bytes) 000273[1] 1739533615.306185 trunc from 0x9fa6ad to 0x6cacfe (0x32f9ae bytes) 000275[1] 1739533615.360794 read 0x2ea837 thru 0x2ec798 (0x1f62 bytes) 000277[1] 1739533615.392910 read 0x17c397 thru 0x17c96b (0x5d5 bytes) 000278[0] 1739533615.404896 mapwrite 0x6c21a5 thru 0x6c2944 (0x7a0 bytes) 000283[1] 1739533615.547465 trunc from 0x6cacff to 0x7119e9 (0x46ceb bytes) 000286[1] 1739533615.624427 read 0x235af0 thru 0x23743b (0x194c bytes) 000287[0] 1739533615.632187 write 0x2d71ae thru 0x2d7d9c (0xbef bytes) 000290[0] 1739533615.694909 write 0x4a03c6 thru 0x4a15de (0x1219 bytes) 000291[0] 1739533615.725551 mapwrite 0x0f6afd thru 0x0f735e (0x862 bytes) 000292[1] 1739533615.786639 read 0x0e78e8 thru 0x0e7acd (0x1e6 bytes) 000293[1] 1739533615.805210 write 0x27cd42 thru 0x27e840 (0x1aff bytes) 000295[1] 1739533615.852823 read 0x370193 thru 0x371659 (0x14c7 bytes) 000297[1] 1739533615.883938 trunc from 0x7119ea to 0x3919c5 (0x380024 bytes) 000298[1] 1739533615.929302 write 0x81bd11 thru 0x81ccfa (0xfea bytes) 000300[0] 1739533615.931495 trunc from 0x81ccfb to 0x730ad3 (0xec227 bytes) 000301[0] 1739533615.975812 write 0x65f9a9 thru 0x6606cd (0xd25 bytes) 000302[1] 1739533616.000443 read 0x346d68 thru 0x34747a (0x713 bytes) 000304[0] 1739533616.039436 mapread 0x285346 thru 0x286138 (0xdf3 bytes) 000305[0] 1739533616.053320 mapread 0x51fcaa thru 0x520738 (0xa8f bytes) 000307[1] 1739533616.075715 read 0x512da4 thru 0x5149a8 (0x1c05 bytes) 000308[0] 1739533616.102220 mapread 0x0459c9 thru 0x046df5 (0x142d bytes) 000309[1] 1739533616.111501 write 0x44710a thru 0x4478e4 (0x7db bytes) 000310[0] 1739533616.128002 read 0x72a493 thru 0x72aa9d (0x60b bytes) 000312[0] 1739533616.159687 mapwrite 0x4c05a4 thru 0x4c0b23 (0x580 bytes) 000313[1] 1739533616.245758 mapwrite 0x596a8e thru 0x596f5a (0x4cd bytes) 000314[1] 1739533616.311127 trunc from 0x730ad4 to 0x061f9c (0x6ceb37 bytes) 000316[1] 1739533616.355540 mapread 0x04f102 thru 0x050423 (0x1322 bytes) 000318[1] 1739533616.378298 trunc from 0x061f9d to 0x502b40 (0x4a0ba4 bytes) 000320[1] 1739533616.418736 trunc from 0x502b41 to 0x9b8832 (0x4b5cf2 bytes) 000322[1] 1739533616.471642 mapread 0x62a612 thru 0x62bda2 (0x1791 bytes) 000327[1] 1739533616.515629 read 0x017dcf thru 0x018e19 (0x104b bytes) 000329[0] 1739533616.559509 mapwrite 0x9d53de thru 0x9d5ea1 (0xac4 bytes) 000334[0] 1739533616.674903 mapread 0x33f2b2 thru 0x34091f (0x166e bytes) 000335[1] 1739533616.682218 mapread 0x753641 thru 0x753f01 (0x8c1 bytes) 000338[0] 1739533616.746767 mapwrite 0x79a75a thru 0x79c180 (0x1a27 bytes) 000339[0] 1739533616.811593 trunc from 0x9d5ea2 to 0x9a4273 (0x31c2e bytes) 000342[1] 1739533616.862618 write 0x662f87 thru 0x664620 (0x169a bytes) 000345[1] 1739533616.924189 trunc from 0x9a4274 to 0x2b705a (0x6ed219 bytes) 000346[1] 1739533616.963439 trunc from 0x2b705b to 0x9ab721 (0x6f46c7 bytes) 000347[0] 1739533616.993197 mapread 0x69bb9a thru 0x69c141 (0x5a8 bytes) 000351[1] 1739533617.049083 mapread 0x2be4b3 thru 0x2be4cf (0x1d bytes) 000352[1] 1739533617.054277 read 0x8aff00 thru 0x8b0efe (0xfff bytes) 000353[0] 1739533617.058374 write 0x53bc49 thru 0x53cafd (0xeb5 bytes) 000354[0] 1739533617.121107 read 0x323bec thru 0x32466b (0xa80 bytes) 000362[1] 1739533617.235773 mapread 0x5a7f3e thru 0x5a93ef (0x14b2 bytes) 000366[1] 1739533617.285756 read 0x64e731 thru 0x64efb0 (0x880 bytes) 000368[0] 1739533617.290986 mapwrite 0x17afec thru 0x17b89e (0x8b3 bytes) 000369[1] 1739533617.374933 mapwrite 0x281be8 thru 0x283b75 (0x1f8e bytes) 000372[0] 1739533617.542265 mapread 0x2b1d40 thru 0x2b1f83 (0x244 bytes) 000374[0] 1739533617.588892 write 0x7dd1fa thru 0x7de7f1 (0x15f8 bytes) 000375[0] 1739533617.632567 mapread 0x234a5c thru 0x2350a6 (0x64b bytes) 000376[0] 1739533617.660595 write 0x856e0e thru 0x857896 (0xa89 bytes) 000379[1] 1739533617.698108 trunc from 0x9ab722 to 0x55796b (0x453db6 bytes) 000381[1] 1739533617.809128 trunc from 0x55796c to 0x2599ac (0x2fdfbf bytes) 000386[0] 1739533617.928688 write 0x4df34a thru 0x4df473 (0x12a bytes) 000387[0] 1739533617.955274 write 0x0d3265 thru 0x0d4ab0 (0x184c bytes) 000391[0] 1739533618.025299 trunc from 0x4df474 to 0x574364 (0x94ef1 bytes) 000396[0] 1739533618.113081 mapread 0x102adf thru 0x104a04 (0x1f26 bytes) 000397[0] 1739533618.141471 mapread 0x0a0cb5 thru 0x0a0fdc (0x328 bytes) 000399[0] 1739533618.169847 trunc from 0x574365 to 0x975821 (0x4014bd bytes) 000400[1] 1739533618.203012 trunc from 0x975822 to 0x682a03 (0x2f2e1e bytes) 000401[1] 1739533618.256692 trunc from 0x682a04 to 0x5974c2 (0xeb541 bytes) 000404[0] 1739533618.298850 write 0x1d895b thru 0x1d9e94 (0x153a bytes) 000405[1] 1739533618.329142 write 0x3acc7d thru 0x3ae996 (0x1d1a bytes) 000407[1] 1739533618.379731 mapwrite 0x47e046 thru 0x47f2e5 (0x12a0 bytes) 000408[0] 1739533618.440336 mapwrite 0x538529 thru 0x53896a (0x442 bytes) 000409[0] 1739533618.535624 write 0x9db6f5 thru 0x9dd56a (0x1e76 bytes) 000412[0] 1739533618.557272 mapread 0x15b4d4 thru 0x15c4b0 (0xfdd bytes) 000418[1] 1739533618.622880 trunc from 0x9dd56b to 0x03bf46 (0x9a1624 bytes) 000421[0] 1739533618.695496 trunc from 0x03bf47 to 0x50d704 (0x4d17be bytes) 000422[0] 1739533618.742682 mapwrite 0x63be61 thru 0x63beed (0x8d bytes) 000424[1] 1739533618.798211 read 0x1d9114 thru 0x1da9ee (0x18db bytes) 000426[0] 1739533618.909029 mapread 0x58f759 thru 0x590be8 (0x1490 bytes) 000429[0] 1739533618.966691 write 0x6108a2 thru 0x6123c5 (0x1b24 bytes) 000433[0] 1739533618.984258 read 0x23a1de thru 0x23b66d (0x1490 bytes) 000434[1] 1739533618.998322 mapwrite 0x1d21c9 thru 0x1d2a25 (0x85d bytes) 000436[1] 1739533619.122976 mapread 0x5cffcc thru 0x5d0aec (0xb21 bytes) 000438[1] 1739533619.213573 read 0x5c05c3 thru 0x5c15f4 (0x1032 bytes) 000439[0] 1739533619.215454 read 0x4db263 thru 0x4dbc28 (0x9c6 bytes) 000440[1] 1739533619.241813 read 0x0c8c11 thru 0x0c9142 (0x532 bytes) 000441[1] 1739533619.265548 mapwrite 0x5eaad9 thru 0x5eb008 (0x530 bytes) 000442[0] 1739533619.379902 mapread 0x11a030 thru 0x11a151 (0x122 bytes) 000443[0] 1739533619.404538 mapwrite 0x1e8b9e thru 0x1e8ced (0x150 bytes) 000444[0] 1739533619.491184 mapwrite 0x2cf012 thru 0x2d0322 (0x1311 bytes) 000454[0] 1739533619.731834 read 0x0d3a34 thru 0x0d3ac6 (0x93 bytes) 000455[1] 1739533619.753529 read 0x05379a thru 0x0553ee (0x1c55 bytes) 000457[1] 1739533619.786001 write 0x8868f1 thru 0x887b7d (0x128d bytes) 000458[1] 1739533619.805786 write 0x7c95f3 thru 0x7cb521 (0x1f2f bytes) 000459[0] 1739533619.814986 trunc from 0x887b7e to 0x4b0a16 (0x3d7167 bytes) 000461[1] 1739533619.895820 write 0x18df5e thru 0x18e9e8 (0xa8b bytes) 000462[0] 1739533619.932515 trunc from 0x4b0a17 to 0x2c7bca (0x1e8e4c bytes) 000463[1] 1739533619.961444 mapwrite 0x2ca03e thru 0x2ca660 (0x623 bytes) 000465[0] 1739533620.076331 trunc from 0x2ca661 to 0x76fd84 (0x4a5724 bytes) 000468[0] 1739533620.148290 trunc from 0x76fd85 to 0x39d354 (0x3d2a30 bytes) 000470[0] 1739533620.185917 mapwrite 0x6a86da thru 0x6aa4cc (0x1df3 bytes) 000471[1] 1739533620.308187 write 0x9106c8 thru 0x911d21 (0x165a bytes) 000472[0] 1739533620.337939 mapwrite 0x3f3dc3 thru 0x3f596a (0x1ba8 bytes) 000474[1] 1739533620.480647 mapread 0x5bfa8d thru 0x5bfbe1 (0x155 bytes) 000475[0] 1739533620.525254 mapwrite 0x047420 thru 0x0489f1 (0x15d2 bytes) 000476[1] 1739533620.655025 write 0x1085d4 thru 0x10880f (0x23c bytes) 000477[1] 1739533620.709768 read 0x4196f6 thru 0x41ad47 (0x1652 bytes) 000478[1] 1739533620.736648 read 0x77f90e thru 0x7817bc (0x1eaf bytes) 000479[1] 1739533620.763546 read 0x62e387 thru 0x62fc1d (0x1897 bytes) 000487[1] 1739533620.837545 write 0x8a6ca3 thru 0x8a85b3 (0x1911 bytes) 000488[1] 1739533620.856206 trunc from 0x911d22 to 0x9f1cc5 (0xdffa4 bytes) 000490[0] 1739533620.933172 write 0x032061 thru 0x033dde (0x1d7e bytes) 000494[1] 1739533621.055368 mapwrite 0x7d3a7b thru 0x7d3a83 (0x9 bytes) 000495[1] 1739533621.124401 read 0x32e51d thru 0x32f396 (0xe7a bytes) 000498[1] 1739533621.172174 trunc from 0x9f1cc6 to 0x1d07ae (0x821517 bytes) 000499[0] 1739533621.220016 mapread 0x0ece63 thru 0x0ee83b (0x19d9 bytes) 000500[1] 1739533621.257017 mapwrite 0x407988 thru 0x40966d (0x1ce6 bytes) All operations completed A-OK! PASS 16b (18s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 06:47:03 (1739533623) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 06:47:06 (1739533626) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.287795 s, 29.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.222751 s, 37.7 MB/s ldlm.namespaces.MGC192.168.204.135@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800ad422000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012dd4c000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800ad422000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012dd4c000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800ad422000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012dd4c000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.68294 s, 18.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.59148 s, 65.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.960002 s, 109 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.02199 s, 34.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.00944 s, 17.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.4428 s, 72.7 MB/s PASS 16d (25s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 06:47:31 (1739533651) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.123592 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114547 s, 9.2 MB/s PASS 16e (4s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 06:47:35 (1739533655) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 06:47:59 (1739533679) TIMEOUT PASS 16g (23s) == sanityn test 16h: mmap read after truncate file ======= 06:48:22 (1739533702) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (3s) == sanityn test 16i: read after truncate file ============ 06:48:25 (1739533705) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 16j: race dio with buffered i/o ========== 06:48:27 (1739533707) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.440357 s, 59.5 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.391043 s, 67.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.22264 s, 21.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.207706 s, 126 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.14908 s, 22.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.356269 s, 73.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.5957 s, 16.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.484281 s, 54.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.677993 s, 38.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.84715 s, 14.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.86883 s, 14.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.788939 s, 33.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.827961 s, 31.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.52433 s, 17.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.60211 s, 16.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.88404 s, 29.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.01502 s, 25.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 2.05454 s, 12.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 2.28163 s, 11.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (19s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 06:48:46 (1739533726) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! + fsx 1 done: rc=0 Fri Feb 14 06:49:08 EST 2025 All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Fri Feb 14 06:49:08 EST 2025 + fsx 3 done: rc=0 Fri Feb 14 06:49:08 EST 2025 + fsx 4 done: rc=0 Fri Feb 14 06:49:08 EST 2025 + fsx 5 done: rc=0 Fri Feb 14 06:49:08 EST 2025 PASS 16k (25s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 06:49:11 (1739533751) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 06:49:15 (1739533755) mmap test1: basic mmap operation (PASS, 0.095175s) mmap test2: MAP_PRIVATE not write back (PASS, 0.061372s) mmap test3: concurrent mmap ops on two nodes (PASS, 9.7796s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.2574s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.092778s) mmap test6: check mmap write/read content on two nodes (PASS, 0.1142s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.37674s) mmap test11: multi-thread shared mmap (PASS, 0.40483s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0565s) PASS 18 (21s) == sanityn test 19: test concurrent uncached read races ========================================================================= 06:49:36 (1739533776) oleg435-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 06:49:37 (1739533777) PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 06:49:40 (1739533780) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 06:49:42 (1739533782) now is 1739533784 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 new atime is 1739533845 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 06:50:46 (1739533846) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 387 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 387 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 30.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234883 387 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 387 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234883 387 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 06:50:49 (1739533849) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 06:50:52 (1739533852) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 06:50:54 (1739533854) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 06:50:55 (1739533855) utime: good mknod times 1739533855 <= 1739533856 <= 1739533856 for /mnt/lustre/f26a utime: good mknod times 1739533855 <= 1739533856 <= 1739533856 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 06:50:58 (1739533858) PASS 26b (5s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 06:51:03 (1739533863) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-02-14 06:51:04.000000000 -0500 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-02-14 06:51:04.000000000 -0500 Birth: - PASS 26c (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 06:51:08 (1739533868) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 06:51:13 (1739533873) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 06:51:16 (1739533876) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg435-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.135@tcp:/lustre 7542784 38912 7497728 1% /mnt/lustre2 PASS 31b (11s) == sanityn test 31r: open-rename(replace) race =========== 06:51:27 (1739533887) -rw-r--r-- 1 root root 0 Feb 14 06:51 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 06:51:32 (1739533892) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13521 Terminated tail -f $tfile PASS 31s (3s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 06:51:35 (1739533895) PASS 31t (3s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 06:51:39 (1739533899) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 06:51:40 (1739533900) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 06:51:41 (1739533901) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 06:51:42 (1739533902) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 06:51:43 (1739533903) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800ad422000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800ad422000.ost_server_uuid in IDLE state after 0 sec oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800ad422000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800ad422000.ost_server_uuid in FULL state after 0 sec oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800ad422000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800ad422000.ost_server_uuid in IDLE state after 0 sec oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800ad422000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800ad422000.ost_server_uuid in FULL state after 0 sec oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800ad422000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800ad422000.ost_server_uuid in IDLE state after 0 sec oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800ad422000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800ad422000.ost_server_uuid in FULL state after 0 sec PASS 34 (88s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 06:53:11 (1739533991) fail_loc=0x80000317 Race attempt 0 Wait for 20150 20302 for 60 sec... - open/close 3003 (time 1739534003.71 total 10.00 last 300.28) total: 4000 open/close in 12.38 seconds: 323.17 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.135@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre 192.168.204.135@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 06:54:15 (1739534055) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg435-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.314035 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.397618 s, 132 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.355827 s, 147 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7494656) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.328808 s, 159 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.372382 s, 141 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.35338 s, 148 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.318052 s, 165 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7494656) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.298717 s, 176 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7496704) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.351805 s, 149 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.34564 s, 152 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.420985 s, 125 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7494656) after_dd(7443456) after(7494656) PASS 36 (227s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 06:58:02 (1739534282) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 - create 7888 (time 1739534293.84 total 10.00 last 788.80) total: 10000 create in 12.66 seconds: 790.06 ops/second PASS 37 (16s) == sanityn test 39a: file mtime does not change after rename ========================================================== 06:58:18 (1739534298) repeat after cancel_lru_locks PASS 39a (1s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 06:58:19 (1739534299) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 06:58:22 (1739534302) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 06:58:25 (1739534305) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 06:58:26 (1739534306) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 06:58:34 (1739534314) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 06:58:41 (1739534321) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 06:58:49 (1739534329) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 06:58:56 (1739534336) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 06:59:03 (1739534343) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 06:59:07 (1739534347) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 06:59:11 (1739534351) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 06:59:16 (1739534356) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 06:59:20 (1739534360) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 06:59:25 (1739534365) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 06:59:30 (1739534370) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 06:59:35 (1739534375) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 06:59:39 (1739534379) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (664s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 07:10:43 (1739535043) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 07:10:47 (1739535047) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 07:10:52 (1739535052) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 07:10:57 (1739535057) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 07:11:01 (1739535061) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 07:11:06 (1739535066) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 07:11:11 (1739535071) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 07:11:15 (1739535075) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 07:11:20 (1739535080) PASS 43a (21s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 07:11:41 (1739535101) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 07:11:46 (1739535106) Conflict PASS 43c (5s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 07:11:51 (1739535111) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 07:11:55 (1739535115) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 07:12:00 (1739535120) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 07:12:05 (1739535125) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 07:12:10 (1739535130) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 07:12:14 (1739535134) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 07:12:15 (1739535135) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (43s) == sanityn test 43k: unlink vs create ==================== 07:12:58 (1739535178) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (382s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 07:19:20 (1739535560) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Feb 14 07:19:23 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 07:19:24 (1739535564) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 07:19:29 (1739535569) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 07:19:33 (1739535573) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 07:19:37 (1739535577) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 07:19:42 (1739535582) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 07:19:46 (1739535586) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 07:19:51 (1739535591) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 07:19:55 (1739535595) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 07:19:56 (1739535596) PASS 45a (24s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 07:20:20 (1739535620) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 07:20:24 (1739535624) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 07:20:28 (1739535628) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 07:20:32 (1739535632) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 07:20:37 (1739535637) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 07:20:41 (1739535641) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 07:20:45 (1739535645) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 07:20:49 (1739535649) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 07:20:49 (1739535649) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (373s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 07:27:03 (1739536023) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 07:27:07 (1739536027) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 07:27:11 (1739536031) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 07:27:15 (1739536035) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 07:27:19 (1739536039) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 07:27:23 (1739536043) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 07:27:28 (1739536048) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 07:27:32 (1739536052) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 07:27:36 (1739536056) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 07:27:37 (1739536057) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 07:27:38 (1739536058) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 07:27:39 (1739536059) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (0s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 07:27:39 (1739536059) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 07:27:40 (1739536060) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 07:27:41 (1739536061) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 07:27:42 (1739536062) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (0s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 07:27:42 (1739536062) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00446821 s, 2.3 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 07:27:49 (1739536069) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00338561 s, 47.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3519: kill: (10983) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 07:27:53 (1739536073) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00292717 s, 350 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169848 s, 61.7 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 07:28:06 (1739536086) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00389644 s, 263 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 07:28:12 (1739536092) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109286 s, 95.9 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 07:28:15 (1739536095) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2453 0x995 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2453 0x995 0x240000400 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 07:28:19 (1739536099) llite.lustre-ffff8800ad422000.xattr_cache=1 llite.lustre-ffff88012dd4c000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:28:19.000000000 -0500 Modify: 2025-02-14 07:28:19.000000000 -0500 Change: 2025-02-14 07:28:19.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:28:24.000000000 -0500 Modify: 2025-02-14 07:28:25.000000000 -0500 Change: 2025-02-14 07:28:25.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:28:30.000000000 -0500 Modify: 2025-02-14 07:28:30.000000000 -0500 Change: 2025-02-14 07:28:30.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:28:30.000000000 -0500 Modify: 2025-02-14 07:28:30.000000000 -0500 Change: 2025-02-14 07:28:30.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:28:35.000000000 -0500 Modify: 2025-02-14 07:28:36.000000000 -0500 Change: 2025-02-14 07:28:36.000000000 -0500 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:28:35.000000000 -0500 Modify: 2025-02-14 07:28:36.000000000 -0500 Change: 2025-02-14 07:28:36.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 07:28:42 (1739536122) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 07:28:49 (1739536129) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 07:28:56 (1739536136) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 07:29:08 (1739536148) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 07:29:17 (1739536157) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1739536157 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 2.90662 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (22s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 07:29:39 (1739536179) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1739536180 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.20946 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (33s) == sanityn test 60: Verify data_version behaviour ======== 07:30:12 (1739536212) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 07:30:14 (1739536214) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 07:30:15 (1739536215) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 07:30:17 (1739536217) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00242458 s, 27.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 07:30:18 (1739536218) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00402988 s, 16.3 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 07:30:19 (1739536219) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 07:30:20 (1739536220) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0038241 s, 17.1 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 07:30:21 (1739536221) llite.lustre-ffff8800ad422000.xattr_cache=1 llite.lustre-ffff88012dd4c000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 07:30:23 (1739536223) llite.lustre-ffff8800ad422000.xattr_cache=1 llite.lustre-ffff88012dd4c000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 07:30:25 (1739536225) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00260624 s, 393 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00276581 s, 370 kB/s 23396: get lock2 23397: get lock1 23396: done 23396: get lock1 23397: done 23397: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 23396: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 23397: cannot get lock2: Resource deadlock avoided 23397: exit rc=0 23396: done 23396: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 07:30:30 (1739536230) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0171707 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101209 s, 104 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4108: 24038 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 07:30:37 (1739536237) opening files: ........ closing files: ........ PASS 76 (15s) == sanityn test 77a: check FIFO NRS policy =============== 07:30:52 (1739536252) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.105269 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111146 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108328 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031869 s, 32.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296218 s, 35.4 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0359105 s, 29.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0309742 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319995 s, 32.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278101 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289864 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414875 s, 25.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0280795 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469143 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436192 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445995 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367054 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419546 s, 25.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804835 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830791 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865206 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873353 s, 120 MB/s 1048576 bytes (1.0 MB) copied, 0.0134271 s, 78.1 MB/s , 0.0217429 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112921 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104847 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223032 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113051 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137341 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01272 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130484 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169714 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146359 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016514 s, 63.5 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 07:30:54 (1739536254) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0953985 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125288 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163227 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155768 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210825 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013201 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199766 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166576 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129194 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396945 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196241 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044469 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324623 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363765 s, 28.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353983 s, 29.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0492237 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326132 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977143 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00796926 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00667742 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00778928 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826026 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00992425 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.010759 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963731 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941188 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102404 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111181 s, 94.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107027 s, 98.0 MB/s , 0.0106997 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111643 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112623 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124297 s, 84.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0937619 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920254 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104326 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105459 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109125 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374338 s, 28.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406958 s, 25.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0409982 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289664 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356637 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039863 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400053 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393965 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462139 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571828 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451983 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569894 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066553 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148094 s, 70.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00729035 s, 144 MB/s , 0.00740549 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856906 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151862 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892642 s, 117 MB/s , 0.00886168 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106234 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122733 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123876 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127957 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129156 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141054 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133397 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162775 s, 64.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 07:30:57 (1739536257) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0942017 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991682 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854773 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937166 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188545 s, 55.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026733 s, 39.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173171 s, 60.6 MB/s , 0.0299189 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029393 s, 35.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390675 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389561 s, 26.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0412943 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362402 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397777 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334184 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313119 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424716 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0126857 s, 82.7 MB/s , 0.0137297 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885647 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012754 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984741 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00998066 s, 105 MB/s , 0.00933636 s, 112 MB/s 1048576 bytes (1.0 MB) copied, 0.0135511 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010355 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102945 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01062 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983548 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107369 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108428 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112933 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115988 s, 90.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0943352 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144814 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146182 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288398 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310611 s, 33.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370688 s, 28.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0406401 s, 25.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0385144 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0451846 s, 23.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0408808 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422241 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535518 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466979 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423578 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451754 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488727 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500789 s, 20.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00889541 s, 118 MB/s , 0.0100282 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00871127 s, 120 MB/s , 0.00863836 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0068603 s, 153 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00803028 s, 131 MB/s 1048576 bytes (1.0 MB) copied, 0.00881007 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985279 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107762 s, 97.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00852841 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010069 s, 104 MB/s , 0.0101967 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104898 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107082 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114654 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119621 s, 87.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0924403 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968959 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010298 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086923 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00663787 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125509 s, 83.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308409 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434755 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429521 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464548 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363441 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500722 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431249 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450592 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483578 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497479 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0517955 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00602366 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877087 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00632967 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098942 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106313 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00558714 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147552 s, 711 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105761 s, 99.1 MB/s , 0.00935612 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00409767 s, 256 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00361927 s, 290 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106363 s, 98.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123111 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011635 s, 90.1 MB/s , 0.0119877 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126273 s, 83.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 07:31:01 (1739536261) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0838832 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00783296 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886484 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372557 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330733 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390991 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341035 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433062 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412486 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445037 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462117 s, 22.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0549897 s, 19.1 MB/s , 0.0473241 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554248 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055533 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0558075 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0647115 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659647 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131913 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123809 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972812 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100693 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132207 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111883 s, 93.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0141675 s, 74.0 MB/s , 0.012105 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107575 s, 97.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127059 s, 82.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0136672 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137268 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126638 s, 82.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0138686 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133236 s, 78.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107108 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992299 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114042 s, 91.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285832 s, 36.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0276403 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371046 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353122 s, 29.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413937 s, 25.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0406789 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378469 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423429 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421238 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449342 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448475 s, 23.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0457212 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477106 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485026 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00721069 s, 145 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0071275 s, 147 MB/s , 0.00734579 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00760168 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116228 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129132 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957969 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956473 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011254 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113763 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117712 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124871 s, 84.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0135342 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132606 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132518 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150717 s, 69.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.109789 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107369 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121924 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114472 s, 91.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339162 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333047 s, 31.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0312795 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359566 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409593 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451096 s, 23.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379062 s, 27.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0490159 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460735 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433663 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382539 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402424 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568398 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00629132 s, 167 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00736053 s, 142 MB/s 1048576 bytes (1.0 MB) copied, 0.0116593 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948327 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842308 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878535 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094716 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925301 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106648 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0114199 s, 91.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00871736 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119592 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128532 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013281 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130411 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126402 s, 83.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 07:31:05 (1739536265) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.35@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0852269 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133882 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128714 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398673 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307485 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414835 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408251 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387532 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046806 s, 22.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0585294 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0622152 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0623779 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462972 s, 22.6 MB/s 1048576 bytes (1.0 MB) copied, 0.048189 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0581854 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594417 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0708978 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00604541 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00512938 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00527718 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946783 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758138 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00799584 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114649 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797341 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944217 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963095 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00988149 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111561 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115028 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0116271 s, 90.2 MB/s , 0.0117408 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117905 s, 88.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0972889 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142031 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152053 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381767 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383848 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444477 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458696 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0455819 s, 23.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0506482 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469133 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505257 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565073 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0628945 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0606907 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0632439 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0540411 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571427 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920965 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738923 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00215923 s, 486 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797848 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00539135 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116682 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953385 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103901 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149712 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106498 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108448 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101895 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0113527 s, 92.4 MB/s , 0.0114633 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115855 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013019 s, 80.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107574 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186014 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206609 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459724 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478594 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487447 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480195 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493924 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465141 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520499 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506604 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0558133 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0607146 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570069 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0555645 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.059837 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597135 s, 17.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170451 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00739674 s, 142 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00645495 s, 162 MB/s , 0.00619411 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802364 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951774 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817422 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832252 s, 126 MB/s 1048576 bytes (1.0 MB) copied, 0.0196452 s, 53.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0227948 s, 46.0 MB/s 1+0 records in 1+0 records out , 0.0103959 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.0202226 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110083 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114899 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0112503 s, 93.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0100895 s, 104 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.119169 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012324 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154063 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146118 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191263 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852743 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023473 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251458 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101728 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907235 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515669 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316993 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541875 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448178 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417543 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043407 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442788 s, 23.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0126487 s, 82.9 MB/s , 0.016205 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833451 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00754505 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975837 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101385 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950139 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172391 s, 60.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113174 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011107 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119007 s, 88.1 MB/s , 0.011758 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123851 s, 84.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0134109 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127094 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123681 s, 84.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 07:31:11 (1739536271) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.12846 s, 131 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00795311 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848971 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108182 s, 96.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123866 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137316 s, 76.4 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0190894 s, 54.9 MB/s , 0.0118381 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659374 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827891 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205335 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104627 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037273 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412723 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051015 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535512 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13199 s, 7.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0100738 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0136346 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01735 s, 60.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0137913 s, 76.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0114416 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202481 s, 51.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126228 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135256 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137691 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142393 s, 73.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0149002 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156475 s, 67.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0150027 s, 69.9 MB/s , 0.0170276 s, 61.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0164371 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018572 s, 56.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.109389 s, 153 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106961 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128546 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114011 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164336 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332553 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413509 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046787 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382441 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378779 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476558 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480092 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519202 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0503065 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0629496 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485127 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.059479 s, 17.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010286 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00718883 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102975 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104682 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986138 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092399 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910587 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00183138 s, 573 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106414 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101146 s, 104 MB/s , 0.00951624 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124467 s, 84.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100652 s, 104 MB/s , 0.0103433 s, 101 MB/s , 0.0104275 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126778 s, 82.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0968527 s, 173 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972736 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0135626 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254639 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456197 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044756 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467396 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487946 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463203 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474239 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403704 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507195 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541833 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487874 s, 21.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552406 s, 19.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0513865 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0625663 s, 16.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00904588 s, 116 MB/s 1048576 bytes (1.0 MB) copied, 0.00912821 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105258 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111708 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133405 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00986227 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.0108734 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190355 s, 551 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124338 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118757 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117459 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123595 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101286 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992645 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106297 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129869 s, 80.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0890625 s, 188 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870662 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865445 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114694 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241004 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380598 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419695 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320761 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421048 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442863 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404707 s, 25.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455299 s, 23.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0503554 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369877 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036984 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406965 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446915 s, 23.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835526 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870906 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158523 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172983 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00746233 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844974 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787867 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826931 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970767 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978517 s, 107 MB/s , 0.00984739 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118217 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120859 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010474 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108659 s, 96.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0127744 s, 82.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (7s) == sanityn test 77g: Change TBF type directly ============ 07:31:18 (1739536278) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (2s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 07:31:20 (1739536280) oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg435-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg435-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 12 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg435-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 12 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg435-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 07:31:24 (1739536284) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 07:31:30 (1739536290) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0954476 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976443 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139139 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106841 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804491 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123672 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273124 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266209 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284702 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267238 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331826 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226638 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00690431 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255362 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885899 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222475 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00795565 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742006 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122587 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114821 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0021862 s, 480 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124698 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101109 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102435 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117668 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109483 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134023 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134709 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0139103 s, 75.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0140314 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013773 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124084 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157953 s, 66.4 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95983 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4373 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 07:32:19 (1739536339) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.21961 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137505 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173801 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271086 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279002 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0354522 s, 29.6 MB/s , 0.034848 s, 30.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0220609 s, 47.5 MB/s , 0.0400312 s, 26.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.175159 s, 6.0 MB/s , 0.175978 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.185708 s, 5.6 MB/s , 0.176772 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188958 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174562 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191574 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178238 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116622 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121615 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122811 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139956 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143026 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136378 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143278 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146613 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014138 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151351 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158999 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0148143 s, 70.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0160117 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154424 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157108 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188152 s, 55.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8377 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9533 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22123 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134765 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147225 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930012 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151325 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146038 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151663 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944526 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119174 s, 88.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.172047 s, 6.1 MB/s , 0.169781 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17969 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.195701 s, 5.4 MB/s 1048576 bytes (1.0 MB) copied, 0.187947 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187609 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188126 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.202662 s, 5.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0085677 s, 122 MB/s , 0.00891552 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828495 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973592 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112642 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107987 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100781 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107449 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106195 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101872 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112153 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113909 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114022 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124439 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120446 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147338 s, 71.2 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0304 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9594 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (129s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 07:34:28 (1739536468) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22216 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133442 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216121 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221885 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191318 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827149 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259172 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205897 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022308 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.175084 s, 6.0 MB/s 1048576 bytes (1.0 MB) copied, 0.17511 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.162534 s, 6.5 MB/s , 0.177687 s, 5.9 MB/s , 0.185016 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176004 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.19759 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174554 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109243 s, 96.0 MB/s , 0.00852485 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114495 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115601 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105489 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106619 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108393 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010551 s, 99.4 MB/s , 0.0132085 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119524 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123529 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136962 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121816 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125841 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142499 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159296 s, 65.8 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0282 s, 5.2 MB/s Write runtime is 22 s, speed is 4.545454 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9653 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg435-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22175 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125463 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959383 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00781148 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194653 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286794 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00608356 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293007 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978763 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.169285 s, 6.2 MB/s , 0.166884 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176352 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176832 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180052 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161969 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184378 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188912 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973999 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904453 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104692 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105856 s, 99.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0116331 s, 90.1 MB/s , 0.0117143 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129463 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107972 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114423 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111807 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0133477 s, 78.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0113844 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0120068 s, 87.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0121012 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129708 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132219 s, 79.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0276 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9578 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (130s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 07:36:38 (1739536598) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.239677 s, 70.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991935 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077573 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847807 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286546 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373984 s, 28.0 MB/s , 0.0289974 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362702 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349149 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451868 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461007 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423977 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416183 s, 25.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0773963 s, 13.5 MB/s , 0.0816092 s, 12.8 MB/s , 0.0817985 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0808877 s, 13.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134617 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178183 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181027 s, 57.9 MB/s , 0.0160843 s, 65.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0171779 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212199 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199627 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167016 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145209 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00246901 s, 425 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156408 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015214 s, 68.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0156987 s, 66.8 MB/s , 0.0172349 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179261 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200154 s, 52.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98487 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74086 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.35@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.35@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.229226 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139444 s, 75.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207807 s, 50.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0128326 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0217693 s, 48.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0158021 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271912 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265267 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124892 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149645 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380264 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444107 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289788 s, 36.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.274742 s, 3.8 MB/s , 0.277582 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.315291 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314235 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921445 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0062308 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801593 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110488 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791413 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103991 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834078 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907242 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948199 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111392 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113234 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113399 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118148 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0122233 s, 85.8 MB/s , 0.0133965 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129586 s, 80.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92349 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73066 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.35@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.230748 s, 72.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008259 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010917 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00718413 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168756 s, 62.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201846 s, 51.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0189508 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129515 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305232 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996308 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455964 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419958 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451958 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524771 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051855 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444942 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363772 s, 28.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101998 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00606725 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00630166 s, 166 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823771 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109213 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804968 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984701 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00554406 s, 189 MB/s 1048576 bytes (1.0 MB) copied, 0.0115498 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100527 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101164 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984994 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105827 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105795 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963982 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118874 s, 88.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92874 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87852 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.429635 s, 39.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929444 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114703 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010128 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226972 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308627 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031379 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291021 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248003 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0746965 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0932628 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0932185 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0888677 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.234047 s, 4.5 MB/s , 0.229546 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.268823 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.271842 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912071 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00746097 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00803632 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114521 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892561 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895857 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0036165 s, 290 MB/s 1048576 bytes (1.0 MB) copied, 0.0095532 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00531153 s, 197 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113635 s, 92.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0121095 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122525 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129149 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129715 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136562 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01414 s, 74.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77805 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73101 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91594 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72846 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.851209 s, 19.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897756 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185156 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932836 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118623 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924035 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147931 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020644 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069107 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155589 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166413 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165659 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169922 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185764 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170379 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.19039 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192024 s, 5.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140985 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111952 s, 93.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0129432 s, 81.0 MB/s , 0.0113785 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125822 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172707 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115773 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123686 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00654499 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128898 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013916 s, 75.4 MB/s , 0.0127934 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132172 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155163 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139614 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166838 s, 62.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7008 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4331 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.239197 s, 70.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150426 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813767 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174831 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201869 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157582 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174226 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195085 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246258 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346145 s, 30.3 MB/s , 0.011077 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357588 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423408 s, 24.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0421313 s, 24.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0338967 s, 30.9 MB/s , 0.0545785 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0533601 s, 19.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0071559 s, 147 MB/s , 0.00703306 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914967 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073594 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883682 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837838 s, 125 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00958507 s, 109 MB/s , 0.0107094 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956615 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101649 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107514 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110246 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104182 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.0127181 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00131784 s, 796 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140606 s, 74.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92313 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7297 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.449073 s, 37.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960016 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116264 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158141 s, 66.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00897744 s, 117 MB/s , 0.00751051 s, 140 MB/s 1048576 bytes (1.0 MB) copied, 0.00911058 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00631508 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00548598 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.075179 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0808178 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0842881 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0884673 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.37974 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.379273 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.431085 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.416274 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00780469 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731949 s, 143 MB/s 1+0 records in 1+0 records out , 0.00671952 s, 156 MB/s 1048576 bytes (1.0 MB) copied, 0.00683031 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772514 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100741 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835087 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813162 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00807591 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00141349 s, 742 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010277 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108726 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934412 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094261 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955016 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107698 s, 97.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78153 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73151 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81147 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74375 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (343s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 07:42:21 (1739536941) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 07:42:24 (1739536944) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg435-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.4658 s, 303 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg435-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0372 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 07:43:14 (1739536994) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0808497 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979035 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107464 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264523 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249714 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296843 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322489 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035498 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378719 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357525 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302697 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298468 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419523 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038029 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392324 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361549 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425821 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911897 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00489809 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00233928 s, 448 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686068 s, 153 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0092987 s, 113 MB/s , 0.00978067 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101852 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742915 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823168 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847675 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921221 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982316 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108344 s, 96.8 MB/s 1048576 bytes (1.0 MB) copied, 0.010416 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108854 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103946 s, 101 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94612 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97351 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106147 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00724785 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909606 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111061 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234689 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248321 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243714 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277846 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293998 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284905 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285145 s, 36.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.215097 s, 4.9 MB/s , 0.213553 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.249044 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.241112 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.28611 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.306595 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018726 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214214 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200064 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211363 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019019 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194721 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203847 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020746 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211006 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210384 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216746 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227875 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227152 s, 46.2 MB/s , 0.0235055 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228941 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229455 s, 45.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg435-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg435-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94701 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96968 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (62s) == sanityn test 77o: Changing rank should not panic ====== 07:44:16 (1739537056) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (2s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 07:44:18 (1739537058) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (27s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 07:44:45 (1739537085) mds.MDS.mdt.nrs_policies=tbf oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg435-client: oleg435-server: ssh exited with exit code 36 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 oleg435-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg435-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 07:44:54 (1739537094) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 21135 kill md_thread xargs PASS 77r (38s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 07:45:32 (1739537132) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 07:45:34 (1739537134) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506690 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:45:34.000000000 -0500 Modify: 2025-02-14 07:45:34.000000000 -0500 Change: 2025-02-14 07:45:35.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 07:45:46 (1739537146) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 07:45:47 (1739537147) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (0s) == sanityn test 81a: rename and stat under striped directory ========================================================== 07:45:47 (1739537147) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 07:45:48 (1739537148) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 07:45:49 (1739537149) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 07:45:50 (1739537150) total: 2500 open/close in 5.03 seconds: 497.18 ops/second snapshot_time 1739537168.846034680 secs.nsecs start_time 1739537151.362961822 secs.nsecs elapsed_time 17.483072858 secs.nsecs open 2505 samples [usecs] 62 1551 656675 185726475 close 2504 samples [usecs] 53 1033 201079 18088499 mknod 2501 samples [usecs] 150 1490 580526 145853988 unlink 2500 samples [usecs] 261 7360 979698 690469472 mkdir 2 samples [usecs] 612 762 1374 955188 rmdir 2 samples [usecs] 1665 2678 4343 9943909 rename 2500 samples [usecs] 224 12486 2164401 2726923665 getattr 3760 samples [usecs] 5 810 64562 4994548 parallel_rename_file 2500 samples [usecs] 224 12486 2164401 2726923665 crossdir_rename 2500 samples [usecs] 224 12486 2164401 2726923665 PASS 81d (19s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 07:46:09 (1739537169) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7021 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7021 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 07:46:11 (1739537171) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 07:46:12 (1739537172) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 85: Lustre API root cache race =========== 07:46:19 (1739537179) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1739537179 Finishing test test1 at 1739537180 PASS 85 (2s) == sanityn test 90: open/create and unlink striped directory ========================================================== 07:46:21 (1739537181) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (0s) == sanityn test 91: chmod and unlink striped directory === 07:46:21 (1739537181) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 07:46:22 (1739537182) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 07:46:23 (1739537183) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5989 0x1765 0x280000400 0 6006 0x1776 0x240000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5990 0x1766 0x280000400 0 6007 0x1777 0x240000400 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 07:46:31 (1739537191) osc.lustre-OST0000-osc-ffff8800ad422000.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012dd4c000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800ad422000.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012dd4c000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0194613 s, 421 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99606 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5744: 9845 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0174819 s, 234 kB/s osc.lustre-OST0000-osc-ffff8800ad422000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012dd4c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800ad422000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012dd4c000.idle_timeout=debug PASS 94 (9s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 07:46:40 (1739537200) llite.lustre-ffff8800ad422000.fast_read=0 llite.lustre-ffff88012dd4c000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0153344 s, 137 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800ad422000.fast_read=1 llite.lustre-ffff88012dd4c000.fast_read=1 PASS 95a (19s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 07:46:59 (1739537219) llite.lustre-ffff8800ad422000.fast_read=0 llite.lustre-ffff88012dd4c000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00169872 s, 7.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800ad422000.fast_read=1 llite.lustre-ffff88012dd4c000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 07:47:07 (1739537227) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 07:47:08 (1739537228) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00195751 s, 2.1 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 07:47:10 (1739537230) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0311325 s, 67.4 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 07:47:12 (1739537232) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0281529 s, 74.5 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 07:47:14 (1739537234) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00270707 s, 4.4 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 07:47:16 (1739537236) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000609995 s, 6.7 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 07:47:18 (1739537238) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000795418 s, 5.1 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 07:47:20 (1739537240) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00102782 s, 4.0 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 07:47:23 (1739537243) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1739537243 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1739537246 Finishing test test23 at 1739537246 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 07:47:30 (1739537250) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 07:47:31 (1739537251) fail_loc=0x416 PASS 105 (28s) == sanityn test 106a: Verify the btime via statx() ======= 07:47:59 (1739537279) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (0s) == sanityn test 106b: Glimpse RPCs test for statx ======== 07:47:59 (1739537279) SKIP: sanityn test_106b statx() only test SKIP 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 07:48:01 (1739537281) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 07:48:02 (1739537282) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0678171 s, 155 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6014 0x177e 0x240000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 07:48:06 (1739537286) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.057313 s, 183 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5996 0x176c 0x280000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7021 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 07:48:14 (1739537294) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00319374 s, 3.1 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.22475 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 07:48:21 (1739537301) 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 3 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 4 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 16 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 20 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 21 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 37 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 40 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 41 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 47 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 50 Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 192.168.204.135@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.135@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg435-client.virtnet /mnt/lustre (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre2 PASS 109 (399s) == sanityn test 110: do not grant another lock on resend ========================================================== 07:55:00 (1739537700) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-14 07:55:00.000000000 -0500 Modify: 2025-02-14 07:55:00.000000000 -0500 Change: 2025-02-14 07:55:00.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg435-client.virtnet oleg435-client.virtnet PASS 110 (166s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 07:57:46 (1739537866) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (0s) == sanityn test 112: update max-inherit in default LMV === 07:57:46 (1739537866) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 07:57:47 (1739537867) lustre-OST0000-osc-ffff88012af66000 active. lustre-OST0001-osc-ffff88012af66000 active. lustre-OST0000-osc-ffff88012af66000 active. lustre-OST0001-osc-ffff88012af66000 active. MGC192.168.204.135@tcp active. PASS 113 (1s) == sanityn test 114: implicit default LMV inherit ======== 07:57:48 (1739537868) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 07:57:49 (1739537869) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 07:57:50 (1739537870) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 07:57:51 (1739537871) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=20 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 20.0152 s, 0.2 kB/s , 20.0119 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1129 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 20.1165 s, 0.2 kB/s , 20.1133 s, 0.2 kB/s , 20.1182 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1201 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.1163 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.118 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1173 s, 0.1 kB/s , 40.1056 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2039 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2115 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 40.1896 s, 0.1 kB/s 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.2058 s, 0.1 kB/s , 40.216 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 40.1916 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 40.211 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2116 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 60.1984 s, 0.1 kB/s , 60.2223 s, 0.1 kB/s , 60.2335 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 60.2351 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2996 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3252 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 60.3119 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 60.3134 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2087 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2555 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3118 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3137 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3185 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3237 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4076 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4259 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.4099 s, 0.1 kB/s , 80.4068 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.353 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.428 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.439 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b3891000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b3891000.ost_server_uuid in IDLE state after 0 sec PASS 200 (104s) cleanup: ====================================================== == sanityn test complete, duration 4698 sec ============== 07:59:36 (1739537976) === sanityn: start cleanup 07:59:36 (1739537976) === Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg435-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 08:00:13 (1739538013) ===