-----============= acceptance-small: sanityn ============----- Thu Apr 17 05:45:49 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg349-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 05:45:53 (1744883153) === Starting client oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Started clients oleg349-client.virtnet: 192.168.203.149@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg349-client.virtnet: executing check_config_client /mnt/lustre oleg349-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg349-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800a9c64000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b5836800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9c64000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5836800.idle_timeout=debug disable quota as required oleg349-server: oleg349-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 05:46:00 (1744883160) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111438 s, 94.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7038] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 05:46:01 (1744883161) PASS 0 (2s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 05:46:03 (1744883163) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 05:46:05 (1744883165) -rw-r--r-- 1 root root 0 Apr 17 05:46 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 05:46:06 (1744883166) -rw-r--r-- 1 root root 0 Apr 17 05:46 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 05:46:08 (1744883168) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 05:46:10 (1744883170) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 05:46:11 (1744883171) total 1 -rwxrwxrwx 1 root root 0 Apr 17 05:46 f2a -rwxrwxrwx 1 root root 0 Apr 17 05:46 f2b total 1 -rwxrwxrwx 1 root root 0 Apr 17 05:46 f2a -rwxrwxrwx 1 root root 0 Apr 17 05:46 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00314375 s, 163 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 05:46:13 (1744883173) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 05:46:14 (1744883174) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0754883 s, 27.8 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 05:46:16 (1744883176) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 05:46:17 (1744883177) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 05:46:20 (1744883180) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 05:46:21 (1744883181) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 05:46:23 (1744883183) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) == sanityn test 8: remove of open special file on other node ==================================================================== 05:46:24 (1744883184) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 05:46:26 (1744883186) PASS 9a (1s) == sanityn test 9b: append to striped sparse file ======== 05:46:27 (1744883187) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014965 s, 70.1 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0164353 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 05:46:29 (1744883189) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00244856 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00207913 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00199353 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00300202 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00281127 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00268869 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00328954 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00393979 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00242694 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00469254 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00211396 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00312119 s, 0.3 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 05:46:31 (1744883191) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00513804 s, 598 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0129058 s, 317 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000424836 s, 7.2 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 05:46:33 (1744883193) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (1s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 05:46:34 (1744883194) warning: '-runtime' deprecated, use '-t runtime' instead - link 1856 (time 1744883205.33 total 10.00 last 185.53) - link 3579 (time 1744883215.34 total 20.01 last 172.24) - link 5386 (time 1744883225.34 total 30.01 last 180.67) - link 7018 (time 1744883235.34 total 40.01 last 163.11) - link 9164 (time 1744883245.34 total 50.02 last 214.57) - link 10000 (time 1744883249.28 total 53.95 last 212.28) - link 11921 (time 1744883259.29 total 63.96 last 192.02) - link 14092 (time 1744883269.29 total 73.96 last 217.03) - link 15752 (time 1744883279.29 total 83.97 last 165.92) - link 17651 (time 1744883289.29 total 93.97 last 189.90) total: 18697 link in 100.00 seconds: 186.96 ops/second using seed 3597661896 running for 100 seconds - stat 10000 (time 1744883202 ; total 6 ; last 6) - stat 20000 (time 1744883207 ; total 11 ; last 5) - stat 30000 (time 1744883213 ; total 17 ; last 6) - stat 40000 (time 1744883218 ; total 22 ; last 5) - stat 50000 (time 1744883224 ; total 28 ; last 6) - stat 60000 (time 1744883229 ; total 33 ; last 5) - stat 70000 (time 1744883235 ; total 39 ; last 6) - stat 80000 (time 1744883240 ; total 44 ; last 5) - stat 90000 (time 1744883246 ; total 50 ; last 6) - stat 97207 (time 1744883257 ; total 61 ; last 11) - stat 100000 (time 1744883261 ; total 65 ; last 4) - stat 107270 (time 1744883272 ; total 76 ; last 11) - stat 110000 (time 1744883277 ; total 81 ; last 5) - stat 116676 (time 1744883288 ; total 92 ; last 11) - stat 120000 (time 1744883294 ; total 98 ; last 6) total: 121631 stats in 100 seconds: 1216.310059 stats/second - unlinked 0 (time 1744883246 ; total 0 ; last 0) - unlinked 10000 (time 1744883312 ; total 66 ; last 66) unlink(/mnt/lustre2/lockdir/lockfile18697) error: No such file or directory total: 18696 unlinks in 94 seconds: 198.893616 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22046) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22049) - No such process PASS 12 (147s) == sanityn test 13: test directory page revocation ======= 05:49:01 (1744883341) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 05:49:03 (1744883343) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 23441 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 05:49:04 (1744883344) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 24074 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 05:49:05 (1744883345) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24706 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 05:49:07 (1744883347) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25341 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 05:49:09 (1744883349) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 25975 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 05:49:11 (1744883351) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 05:49:23 (1744883363) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.137068 s, 61.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.105939 s, 79.2 MB/s Chance of close/open is 1 in 50 Seed set to 3364 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1744883364.399427 mapwrite 0x890b9c thru 0x894576 (0x39db bytes) 000002[1] 1744883364.445282 mapread 0x4ce763 thru 0x4da512 (0xbdb0 bytes) 000003[1] 1744883364.456710 mapwrite 0x17f673 thru 0x18c694 (0xd022 bytes) 000004[1] 1744883364.484140 write 0x2b3a0c thru 0x2b3c4f (0x244 bytes) 000005[0] 1744883364.485924 mapwrite 0x2e8753 thru 0x2e8b36 (0x3e4 bytes) 000007[1] 1744883364.525729 mapwrite 0x2b6d89 thru 0x2c2a83 (0xbcfb bytes) truncating to largest ever: 0x85165d 000009[1] 1744883364.568339 trunc from 0x894577 to 0x85165c (0x42f1a bytes) 000011[0] 1744883364.576636 read 0x49a7fd thru 0x4a9509 (0xed0d bytes) 000012[0] 1744883364.583515 mapwrite 0x8b09c7 thru 0x8bd259 (0xc893 bytes) 000014[0] 1744883364.629425 write 0x294839 thru 0x2973db (0x2ba3 bytes) 000016[1] 1744883364.637412 mapwrite 0x58ecee thru 0x592404 (0x3717 bytes) 000019[1] 1744883364.673388 trunc from 0x8bd25a to 0x5317f7 (0x38ba62 bytes) 000021[1] 1744883364.689926 read 0x386c1b thru 0x38f4ef (0x88d5 bytes) 000023[0] 1744883364.697128 mapread 0x5286e9 thru 0x52f76e (0x7086 bytes) 000024[0] 1744883364.710398 mapread 0x3a104a thru 0x3ad9a3 (0xc95a bytes) 000025[1] 1744883364.726527 write 0x787d1b thru 0x792b77 (0xae5d bytes) truncating to largest ever: 0x9e48e0 000027[1] 1744883364.746799 trunc from 0x792b78 to 0x9e48df (0x251d68 bytes) 000030[0] 1744883364.767110 mapwrite 0x530766 thru 0x5380ac (0x7947 bytes) 000031[0] 1744883364.800499 read 0x4971cd thru 0x4a21c2 (0xaff6 bytes) 000032[0] 1744883364.807444 write 0x2190c3 thru 0x224926 (0xb864 bytes) 000033[0] 1744883364.812670 read 0x4ef01a thru 0x4f5ae4 (0x6acb bytes) 000039[1] 1744883364.833948 mapread 0x7527a7 thru 0x757d1f (0x5579 bytes) 000040[0] 1744883364.849787 read 0x849baa thru 0x858042 (0xe499 bytes) 000041[0] 1744883364.857162 trunc from 0x9e48e0 to 0x5a9119 (0x43b7c6 bytes) 000042[0] 1744883364.875731 trunc from 0x5a911a to 0x697ea5 (0xeed8c bytes) 000046[0] 1744883364.906994 mapwrite 0x289349 thru 0x2935ee (0xa2a6 bytes) 000047[1] 1744883364.927020 mapread 0x5595b1 thru 0x55e702 (0x5152 bytes) 000049[1] 1744883364.939099 mapwrite 0x06aafb thru 0x072dc9 (0x82cf bytes) 000050[0] 1744883364.967176 mapwrite 0x71d2d8 thru 0x721456 (0x417f bytes) 000051[0] 1744883365.005132 mapwrite 0x26a14e thru 0x278307 (0xe1ba bytes) 000053[1] 1744883365.045635 mapread 0x3115c6 thru 0x320ba0 (0xf5db bytes) 000054[0] 1744883365.065914 read 0x36c0d4 thru 0x37b271 (0xf19e bytes) 000056[1] 1744883365.072738 mapread 0x127c22 thru 0x131e24 (0xa203 bytes) 000063[0] 1744883365.095294 write 0x8ba79e thru 0x8c87e6 (0xe049 bytes) 000065[1] 1744883365.105738 trunc from 0x8c87e7 to 0x2daf61 (0x5ed885 bytes) 000067[1] 1744883365.126966 read 0x1e37de thru 0x1e8850 (0x5073 bytes) 000068[0] 1744883365.153466 mapread 0x07c770 thru 0x08b585 (0xee16 bytes) 000070[1] 1744883365.178533 trunc from 0x2daf62 to 0x3ecced (0x111d8c bytes) 000071[0] 1744883365.189864 trunc from 0x3eccee to 0x23db25 (0x1af1c8 bytes) 000073[1] 1744883365.209383 read 0x1660b7 thru 0x16766a (0x15b4 bytes) 000079[1] 1744883365.246433 mapwrite 0x99ac71 thru 0x99da15 (0x2da5 bytes) 000081[0] 1744883365.351238 write 0x272380 thru 0x280a05 (0xe686 bytes) 000082[0] 1744883365.366520 trunc from 0x99da16 to 0x60b65a (0x3923bb bytes) 000088[1] 1744883365.389251 trunc from 0x60b65b to 0x1f698d (0x414ccd bytes) 000090[0] 1744883365.412327 mapwrite 0x86f677 thru 0x87dd9d (0xe727 bytes) 000092[0] 1744883365.458897 read 0x321f3d thru 0x324c6e (0x2d32 bytes) 000093[0] 1744883365.477660 mapread 0x04daa4 thru 0x04fc0a (0x2167 bytes) 000094[1] 1744883365.491590 write 0x2ce99b thru 0x2dae13 (0xc479 bytes) 000095[0] 1744883365.505646 read 0x282e5c thru 0x28ff6d (0xd112 bytes) 000099[1] 1744883365.523473 mapwrite 0x443cdb thru 0x44a634 (0x695a bytes) 000103[0] 1744883365.568742 trunc from 0x87dd9e to 0x5b64c3 (0x2c78da bytes) 000104[1] 1744883365.585263 write 0x3e4cca thru 0x3ecd25 (0x805c bytes) 000106[0] 1744883365.610534 mapread 0x3b9fc8 thru 0x3bdef7 (0x3f30 bytes) 000108[0] 1744883365.640836 trunc from 0x5b64c4 to 0x416fb2 (0x19f511 bytes) 000110[0] 1744883365.659738 mapread 0x2618dd thru 0x26b606 (0x9d2a bytes) 000111[0] 1744883365.671787 mapwrite 0x1fb8dd thru 0x2040b2 (0x87d6 bytes) 000112[1] 1744883365.707297 trunc from 0x416fb3 to 0x4cadc0 (0xb3e0e bytes) 000113[0] 1744883365.729706 mapread 0x1bd076 thru 0x1bf547 (0x24d2 bytes) 000115[1] 1744883365.743158 mapread 0x2e730e thru 0x2e8888 (0x157b bytes) 000116[0] 1744883365.746323 write 0x466efb thru 0x4702f8 (0x93fe bytes) 000117[0] 1744883365.757841 trunc from 0x4cadc1 to 0x69eec9 (0x1d4109 bytes) 000118[0] 1744883365.775081 write 0x97b96e thru 0x97c840 (0xed3 bytes) 000119[1] 1744883365.778370 trunc from 0x97c841 to 0x97a135 (0x270b bytes) 000120[0] 1744883365.802108 mapwrite 0x58bd74 thru 0x58c94a (0xbd7 bytes) 000124[0] 1744883365.855131 read 0x0255b5 thru 0x029e81 (0x48cd bytes) 000128[0] 1744883365.874000 mapread 0x33d677 thru 0x34a9e4 (0xd36e bytes) 000129[0] 1744883365.887595 read 0x847a6b thru 0x848533 (0xac9 bytes) 000130[0] 1744883365.894757 write 0x7a0eef thru 0x7a3b40 (0x2c52 bytes) 000131[1] 1744883365.898277 read 0x224953 thru 0x22afd4 (0x6682 bytes) 000133[0] 1744883365.901628 mapread 0x2af4c7 thru 0x2b06e0 (0x121a bytes) 000134[0] 1744883365.908879 read 0x467e7c thru 0x475f24 (0xe0a9 bytes) 000135[0] 1744883365.918631 read 0x44b712 thru 0x45441e (0x8d0d bytes) 000139[1] 1744883365.933576 write 0x700ccb thru 0x705ada (0x4e10 bytes) 000140[0] 1744883365.944834 trunc from 0x97a136 to 0x2be040 (0x6bc0f5 bytes) 000144[0] 1744883365.971515 mapread 0x12fa27 thru 0x130b3d (0x1117 bytes) 000145[1] 1744883365.974494 write 0x32226f thru 0x327cea (0x5a7c bytes) 000146[1] 1744883365.982026 trunc from 0x327ceb to 0x2ee101 (0x39be9 bytes) 000147[1] 1744883365.991191 write 0x59dace thru 0x5a6919 (0x8e4c bytes) 000148[1] 1744883365.999164 trunc from 0x5a691a to 0x399aed (0x20ce2c bytes) 000149[0] 1744883366.005858 mapread 0x07e3ab thru 0x085115 (0x6d6b bytes) 000152[0] 1744883366.016636 trunc from 0x399aee to 0x1e5e7f (0x1b3c6e bytes) 000156[0] 1744883366.039131 mapwrite 0x3c8523 thru 0x3d4985 (0xc463 bytes) 000159[0] 1744883366.086908 read 0x0ecfeb thru 0x0f16e8 (0x46fe bytes) 000160[0] 1744883366.114543 trunc from 0x3d4986 to 0x346fe7 (0x8d99e bytes) 000162[1] 1744883366.137799 mapread 0x0239c0 thru 0x025d23 (0x2364 bytes) 000163[0] 1744883366.155796 read 0x0a7185 thru 0x0b5a88 (0xe904 bytes) 000164[1] 1744883366.178445 mapwrite 0x3dd1bd thru 0x3df734 (0x2578 bytes) 000167[0] 1744883366.217345 mapread 0x2daa20 thru 0x2e75a5 (0xcb86 bytes) 000168[0] 1744883366.253744 read 0x2a91f8 thru 0x2b3610 (0xa419 bytes) 000169[0] 1744883366.279688 mapread 0x29e8a0 thru 0x2a336a (0x4acb bytes) 000170[1] 1744883366.283431 mapwrite 0x3d4255 thru 0x3dc981 (0x872d bytes) 000171[0] 1744883366.317638 mapwrite 0x2aba21 thru 0x2b0949 (0x4f29 bytes) 000172[1] 1744883366.356444 trunc from 0x3df735 to 0x38a592 (0x551a2 bytes) 000174[0] 1744883366.365010 mapwrite 0x4db3b8 thru 0x4e02f8 (0x4f41 bytes) 000176[1] 1744883366.414308 trunc from 0x4e02f9 to 0x2f8315 (0x1e7fe3 bytes) 000178[1] 1744883366.429477 read 0x2c4787 thru 0x2d4157 (0xf9d1 bytes) 000179[0] 1744883366.456522 trunc from 0x2f8316 to 0x044798 (0x2b3b7d bytes) 000180[1] 1744883366.471624 write 0x231b24 thru 0x238178 (0x6655 bytes) 000181[1] 1744883366.479313 mapwrite 0x911a1f thru 0x9156e3 (0x3cc5 bytes) 000182[1] 1744883366.544614 trunc from 0x9156e4 to 0x3729e7 (0x5a2cfc bytes) 000184[1] 1744883366.563441 read 0x219c1b thru 0x2202e9 (0x66cf bytes) 000185[0] 1744883366.586900 mapwrite 0x196b42 thru 0x1a4d21 (0xe1e0 bytes) 000187[1] 1744883366.642988 mapwrite 0x5e8f3a thru 0x5ecae4 (0x3bab bytes) 000189[0] 1744883366.698163 write 0x73d11a thru 0x73f8e7 (0x27ce bytes) 000190[0] 1744883366.708173 read 0x0b415b thru 0x0ba6fb (0x65a1 bytes) 000193[0] 1744883366.720349 trunc from 0x73f8e8 to 0x0d3216 (0x66c6d1 bytes) 000195[1] 1744883366.735494 mapread 0x095d26 thru 0x0a56e7 (0xf9c2 bytes) 000197[1] 1744883366.765153 trunc from 0x0d3217 to 0x2aca4f (0x1d9839 bytes) 000202[0] 1744883366.789029 write 0x53c0fd thru 0x548784 (0xc688 bytes) 000203[1] 1744883366.797611 mapwrite 0x2b414c thru 0x2b52cd (0x1182 bytes) 000205[1] 1744883366.841223 write 0x3d5f77 thru 0x3e0dc6 (0xae50 bytes) 000207[1] 1744883366.848287 trunc from 0x548785 to 0x014b58 (0x533c2c bytes) 000208[0] 1744883366.857981 trunc from 0x014b59 to 0x161c79 (0x14d121 bytes) 000213[1] 1744883366.887119 mapread 0x067911 thru 0x069bd7 (0x22c7 bytes) 000216[0] 1744883366.903137 trunc from 0x161c7a to 0x853c53 (0x6f1fda bytes) 000217[1] 1744883366.912156 write 0x0b4060 thru 0x0bc459 (0x83fa bytes) 000219[0] 1744883366.925382 read 0x1b718d thru 0x1bfe17 (0x8c8b bytes) 000222[1] 1744883366.935434 read 0x2543d3 thru 0x25a240 (0x5e6e bytes) 000223[0] 1744883366.943450 write 0x73e1c8 thru 0x744c0e (0x6a47 bytes) 000228[1] 1744883366.963329 trunc from 0x853c54 to 0x52b557 (0x3286fc bytes) 000229[1] 1744883366.975089 write 0x875df2 thru 0x881064 (0xb273 bytes) 000231[0] 1744883366.976772 trunc from 0x881065 to 0x163039 (0x71e02b bytes) 000232[1] 1744883366.997304 read 0x011c86 thru 0x01d3da (0xb755 bytes) 000234[0] 1744883367.021294 write 0x2a1531 thru 0x2aab58 (0x9628 bytes) 000238[0] 1744883367.033984 trunc from 0x2aab59 to 0x4021f7 (0x15769f bytes) 000242[1] 1744883367.054190 mapread 0x09d4ac thru 0x0a4953 (0x74a8 bytes) 000243[0] 1744883367.091427 read 0x15c42a thru 0x16b77b (0xf352 bytes) 000244[1] 1744883367.099277 mapread 0x0c9b97 thru 0x0ce02f (0x4499 bytes) 000245[1] 1744883367.114160 mapwrite 0x9010fd thru 0x90960f (0x8513 bytes) 000246[1] 1744883367.169237 mapwrite 0x410923 thru 0x41a241 (0x991f bytes) 000247[1] 1744883367.198453 write 0x9e134f thru 0x9e1dcb (0xa7d bytes) 000250[1] 1744883367.220473 mapwrite 0x6dd49a thru 0x6e03f2 (0x2f59 bytes) 000252[1] 1744883367.255362 mapread 0x373f98 thru 0x3772ad (0x3316 bytes) 000253[1] 1744883367.269543 write 0x525ae0 thru 0x534020 (0xe541 bytes) 000254[0] 1744883367.270498 read 0x29abf9 thru 0x29b735 (0xb3d bytes) 000257[0] 1744883367.283214 mapwrite 0x698717 thru 0x6a1dae (0x9698 bytes) 000258[1] 1744883367.329653 trunc from 0x9e1dcc to 0x9365dc (0xab7ef bytes) 000260[0] 1744883367.343627 mapwrite 0x32b88a thru 0x32bbff (0x376 bytes) 000262[0] 1744883367.374758 read 0x8e20bb thru 0x8e9e76 (0x7dbc bytes) 000263[0] 1744883367.381028 mapread 0x529a2a thru 0x539610 (0xfbe7 bytes) 000264[0] 1744883367.398365 trunc from 0x9365dd to 0x8a6973 (0x8fc69 bytes) 000266[0] 1744883367.407225 write 0x1fff60 thru 0x20450d (0x45ae bytes) 000268[1] 1744883367.415898 mapread 0x15f731 thru 0x16d440 (0xdd10 bytes) 000269[1] 1744883367.439891 mapread 0x235fef thru 0x2442ed (0xe2ff bytes) 000270[0] 1744883367.479430 trunc from 0x8a6974 to 0x4d90f1 (0x3cd882 bytes) 000271[0] 1744883367.490371 mapwrite 0x98147d thru 0x98874c (0x72d0 bytes) 000272[1] 1744883367.535669 write 0x01ed41 thru 0x0238be (0x4b7e bytes) 000274[0] 1744883367.555736 read 0x10038e thru 0x10e2dc (0xdf4f bytes) 000275[1] 1744883367.567585 trunc from 0x98874d to 0x5b5c4c (0x3d2b00 bytes) 000279[1] 1744883367.596206 trunc from 0x5b5c4d to 0x796608 (0x1e09bc bytes) 000280[0] 1744883367.611357 mapread 0x66ae4b thru 0x66f9a1 (0x4b57 bytes) 000285[1] 1744883367.651354 mapwrite 0x547344 thru 0x54809a (0xd57 bytes) 000287[1] 1744883367.674508 mapread 0x4f1291 thru 0x4f935f (0x80cf bytes) 000289[0] 1744883367.693478 write 0x28f259 thru 0x29ab60 (0xb908 bytes) 000290[0] 1744883367.704128 read 0x0bdb7a thru 0x0cb62f (0xdab6 bytes) 000292[1] 1744883367.715573 write 0x8cccb5 thru 0x8d83aa (0xb6f6 bytes) 000293[1] 1744883367.722917 trunc from 0x8d83ab to 0x93bb05 (0x6375b bytes) 000294[0] 1744883367.729632 trunc from 0x93bb06 to 0x8e5bd6 (0x55f2f bytes) 000298[1] 1744883367.761302 mapwrite 0x43c355 thru 0x4416a7 (0x5353 bytes) 000299[0] 1744883367.800333 write 0x75d34e thru 0x763dfd (0x6ab0 bytes) 000301[1] 1744883367.816474 trunc from 0x8e5bd7 to 0x80e547 (0xd768f bytes) 000302[0] 1744883367.829161 mapread 0x7334a9 thru 0x736473 (0x2fcb bytes) 000303[0] 1744883367.840156 read 0x1f2ece thru 0x1f3414 (0x547 bytes) 000305[0] 1744883367.852602 trunc from 0x80e548 to 0x794cd0 (0x79877 bytes) 000307[1] 1744883367.890999 read 0x69e795 thru 0x6a2e38 (0x46a4 bytes) 000309[1] 1744883367.910450 mapwrite 0x37fa84 thru 0x38bb8d (0xc10a bytes) 000311[1] 1744883367.944460 trunc from 0x794cd1 to 0x80c969 (0x77c99 bytes) 000313[0] 1744883367.953261 trunc from 0x80c96a to 0x4874c0 (0x3854a9 bytes) 000316[0] 1744883367.973484 mapwrite 0x59b250 thru 0x59d7c4 (0x2575 bytes) 000317[1] 1744883368.010181 read 0x11a6ef thru 0x11a992 (0x2a4 bytes) 000318[0] 1744883368.015862 write 0x839a0f thru 0x845ce9 (0xc2db bytes) 000319[0] 1744883368.025468 write 0x5e30a6 thru 0x5e9c0c (0x6b67 bytes) 000320[1] 1744883368.029461 mapwrite 0x3432cc thru 0x34f00f (0xbd44 bytes) 000321[0] 1744883368.080206 trunc from 0x845cea to 0x4500ea (0x3f5bff bytes) 000322[1] 1744883368.089299 trunc from 0x4500eb to 0x19a237 (0x2b5eb3 bytes) 000324[0] 1744883368.103714 mapwrite 0x45623d thru 0x45a8bf (0x4683 bytes) 000326[0] 1744883368.145770 trunc from 0x45a8c0 to 0x86c49d (0x411bde bytes) 000335[1] 1744883368.180365 write 0x08f436 thru 0x096638 (0x7203 bytes) 000337[0] 1744883368.190288 mapwrite 0x11e0ea thru 0x12c4d7 (0xe3ee bytes) 000338[1] 1744883368.222364 write 0x5fcfaf thru 0x5ffacc (0x2b1e bytes) 000339[0] 1744883368.233322 trunc from 0x86c49e to 0x57ac2f (0x2f186e bytes) 000341[0] 1744883368.252011 mapread 0x068b22 thru 0x0787a4 (0xfc83 bytes) 000342[1] 1744883368.261454 mapread 0x41f0a6 thru 0x4248dc (0x5837 bytes) 000343[1] 1744883368.272442 read 0x4f31c0 thru 0x4fc27e (0x90bf bytes) 000344[1] 1744883368.278703 mapread 0x4f3aa0 thru 0x4fd9c8 (0x9f29 bytes) 000346[1] 1744883368.287798 trunc from 0x57ac30 to 0x6e342c (0x1687fd bytes) 000347[1] 1744883368.304520 trunc from 0x6e342d to 0x4775c1 (0x26be6b bytes) 000348[1] 1744883368.314431 mapread 0x08ea79 thru 0x09c83b (0xddc3 bytes) 000349[1] 1744883368.330564 write 0x20638e thru 0x20f8ab (0x951e bytes) 000350[1] 1744883368.334160 write 0x91455c thru 0x91af9f (0x6a44 bytes) 000352[0] 1744883368.343244 mapread 0x43b3b4 thru 0x44a8a1 (0xf4ee bytes) 000360[1] 1744883368.384471 mapwrite 0x372247 thru 0x382096 (0xfe50 bytes) 000363[1] 1744883368.418921 mapwrite 0x821638 thru 0x82ab10 (0x94d9 bytes) 000364[0] 1744883368.440854 read 0x15b9e4 thru 0x1614bc (0x5ad9 bytes) 000367[0] 1744883368.448019 read 0x6c6db9 thru 0x6d198b (0xabd3 bytes) 000370[1] 1744883368.459861 write 0x5d1b5b thru 0x5da983 (0x8e29 bytes) 000379[0] 1744883368.482802 write 0x2b2273 thru 0x2b8ff3 (0x6d81 bytes) 000381[1] 1744883368.494485 mapwrite 0x40a4cc thru 0x40f67f (0x51b4 bytes) 000384[1] 1744883368.523608 write 0x28f98e thru 0x293564 (0x3bd7 bytes) 000386[0] 1744883368.530619 trunc from 0x91afa0 to 0x71b640 (0x1ff95f bytes) 000387[1] 1744883368.541167 mapwrite 0x173a4a thru 0x178b87 (0x513e bytes) 000390[1] 1744883368.573904 write 0x8b1b2e thru 0x8c024a (0xe71d bytes) 000395[0] 1744883368.591814 read 0x158104 thru 0x160e5a (0x8d57 bytes) 000396[0] 1744883368.602312 mapread 0x8709ab thru 0x87d6ed (0xcd43 bytes) 000397[0] 1744883368.608563 read 0x31a0c8 thru 0x321248 (0x7181 bytes) 000398[1] 1744883368.610901 mapread 0x64230c thru 0x647108 (0x4dfd bytes) 000401[0] 1744883368.620304 trunc from 0x8c024b to 0x780724 (0x13fb26 bytes) 000405[0] 1744883368.641918 mapread 0x6ee106 thru 0x6f3849 (0x5744 bytes) 000407[0] 1744883368.650580 mapwrite 0x97ff69 thru 0x9840e6 (0x417e bytes) 000409[0] 1744883368.679469 trunc from 0x9840e7 to 0x52e77e (0x455968 bytes) 000410[1] 1744883368.684508 trunc from 0x52e77f to 0x0e2274 (0x44c50a bytes) 000412[0] 1744883368.697516 mapwrite 0x26e1e8 thru 0x26e834 (0x64d bytes) 000413[0] 1744883368.726273 mapwrite 0x89cba6 thru 0x8a11b7 (0x4612 bytes) 000414[0] 1744883368.746234 write 0x7649dd thru 0x76ee9f (0xa4c3 bytes) 000415[1] 1744883368.753583 write 0x4dbdc2 thru 0x4dfd14 (0x3f53 bytes) 000417[1] 1744883368.767331 mapread 0x2d7426 thru 0x2dc359 (0x4f34 bytes) 000418[1] 1744883368.781297 mapwrite 0x648037 thru 0x648c4c (0xc16 bytes) 000419[0] 1744883368.799581 write 0x801590 thru 0x8027ab (0x121c bytes) 000423[0] 1744883368.818566 mapwrite 0x1f4f0e thru 0x1feea7 (0x9f9a bytes) 000427[1] 1744883368.853625 read 0x7aeec9 thru 0x7b530c (0x6444 bytes) 000428[1] 1744883368.858615 mapwrite 0x85357c thru 0x85d296 (0x9d1b bytes) 000429[0] 1744883368.881661 mapwrite 0x263c39 thru 0x26fcde (0xc0a6 bytes) 000430[1] 1744883368.908292 write 0x10073b thru 0x10a1ec (0x9ab2 bytes) 000432[1] 1744883368.913440 read 0x02165f thru 0x02829d (0x6c3f bytes) 000434[0] 1744883368.918149 write 0x0312ea thru 0x03a9a0 (0x96b7 bytes) 000438[0] 1744883368.935966 write 0x3f6c9c thru 0x4037e3 (0xcb48 bytes) 000440[0] 1744883368.943974 mapread 0x384623 thru 0x3852f1 (0xccf bytes) 000441[0] 1744883368.949408 read 0x113a89 thru 0x114f5c (0x14d4 bytes) 000442[1] 1744883368.953436 write 0x55f6b8 thru 0x561a05 (0x234e bytes) 000445[1] 1744883368.964803 mapwrite 0x5b8eb8 thru 0x5c123a (0x8383 bytes) 000446[1] 1744883368.982139 write 0x286b04 thru 0x287fdd (0x14da bytes) 000448[1] 1744883368.987446 mapread 0x0b9ced thru 0x0c16ed (0x7a01 bytes) 000450[0] 1744883368.991519 write 0x30f687 thru 0x31949d (0x9e17 bytes) 000454[1] 1744883369.009727 mapread 0x023213 thru 0x02d1b4 (0x9fa2 bytes) 000456[0] 1744883369.024087 read 0x2e70b0 thru 0x2e71d5 (0x126 bytes) 000457[1] 1744883369.025608 mapread 0x85516f thru 0x85e86b (0x96fd bytes) 000463[1] 1744883369.050237 mapread 0x4735f1 thru 0x4828b1 (0xf2c1 bytes) 000464[0] 1744883369.057318 write 0x1d8ff2 thru 0x1de712 (0x5721 bytes) 000465[1] 1744883369.065382 write 0x69d9af thru 0x6ac80c (0xee5e bytes) 000466[0] 1744883369.071853 read 0x0f72be thru 0x10613f (0xee82 bytes) 000467[1] 1744883369.078097 read 0x765f01 thru 0x76c2cf (0x63cf bytes) 000469[0] 1744883369.086927 read 0x691a2d thru 0x69c85f (0xae33 bytes) 000471[1] 1744883369.103286 write 0x51c5a4 thru 0x527453 (0xaeb0 bytes) 000476[1] 1744883369.127703 mapwrite 0x21e0ab thru 0x221e89 (0x3ddf bytes) 000477[0] 1744883369.153554 write 0x669375 thru 0x670e0d (0x7a99 bytes) 000479[0] 1744883369.166307 mapwrite 0x65546e thru 0x65bb93 (0x6726 bytes) 000480[0] 1744883369.188488 mapread 0x7d2cb0 thru 0x7e0cd2 (0xe023 bytes) 000481[0] 1744883369.197236 write 0x3a3681 thru 0x3aa616 (0x6f96 bytes) 000483[1] 1744883369.203985 mapread 0x6ccdb7 thru 0x6daba6 (0xddf0 bytes) 000485[0] 1744883369.215900 trunc from 0x8a11b8 to 0x2f6c1d (0x5aa59a bytes) 000486[1] 1744883369.225395 mapread 0x2ccd24 thru 0x2d24e5 (0x57c2 bytes) 000487[1] 1744883369.234980 mapread 0x0d27f0 thru 0x0e0de9 (0xe5fa bytes) 000488[0] 1744883369.244640 trunc from 0x2f6c1e to 0x1bac85 (0x13bf98 bytes) 000495[0] 1744883369.259836 read 0x04d323 thru 0x053edf (0x6bbd bytes) 000500[0] 1744883369.284519 mapread 0x07514f thru 0x083077 (0xdf29 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 3369 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x193947 000010[1] 1744883369.541373 trunc from 00000000 to 0x193946 (0x193947 bytes) truncating to largest ever: 0x4c8e0c 000012[1] 1744883369.551451 trunc from 0x193947 to 0x4c8e0b (0x3354c5 bytes) 000017[1] 1744883369.569510 mapread 0x279000 thru 0x27d7d7 (0x47d8 bytes) 000019[0] 1744883369.573555 write 0x1a3000 thru 0x1abeda (0x8edb bytes) 000021[1] 1744883369.577027 write 0x5fb000 thru 0x604364 (0x9365 bytes) 000022[0] 1744883369.584603 write 0x9ef000 thru 0x9ef8ae (0x8af bytes) 000024[0] 1744883369.602372 read 0x7db000 thru 0x7dc81c (0x181d bytes) truncating to largest ever: 0x544f18 000028[1] 1744883369.608324 trunc from 0x9ef8af to 0x544f17 (0x4aa997 bytes) 000036[1] 1744883369.627023 mapwrite 0x66d000 thru 0x66e67f (0x1680 bytes) 000038[1] 1744883369.652656 read 0x186000 thru 0x187a61 (0x1a62 bytes) 000039[0] 1744883369.654240 mapwrite 0x18a000 thru 0x18a874 (0x875 bytes) 000041[1] 1744883369.677447 mapwrite 0x87f000 thru 0x88871b (0x971c bytes) 000045[1] 1744883369.712230 mapread 0x118000 thru 0x1241b7 (0xc1b8 bytes) 000046[0] 1744883369.719793 mapread 0x2b6000 thru 0x2bc931 (0x6932 bytes) 000047[1] 1744883369.729366 write 0x49e000 thru 0x4ab9bf (0xd9c0 bytes) 000050[1] 1744883369.737473 read 0x42f000 thru 0x436191 (0x7192 bytes) truncating to largest ever: 0x5c2dd4 000051[0] 1744883369.743938 trunc from 0x88871c to 0x5c2dd3 (0x2c5948 bytes) 000054[0] 1744883369.754160 trunc from 0x5c2dd4 to 0x4be9b6 (0x10441d bytes) 000055[1] 1744883369.757770 read 0x2c1000 thru 0x2cbb19 (0xab1a bytes) 000060[0] 1744883369.765223 mapread 0x033000 thru 0x037966 (0x4967 bytes) 000062[0] 1744883369.769615 write 0x7ab000 thru 0x7b202b (0x702c bytes) 000064[1] 1744883369.773701 read 0x20c000 thru 0x21744a (0xb44b bytes) 000065[1] 1744883369.779549 write 0x5fe000 thru 0x60afbc (0xcfbd bytes) 000070[0] 1744883369.790191 read 0x62c000 thru 0x62e2b0 (0x22b1 bytes) 000071[0] 1744883369.795365 mapwrite 0x38d000 thru 0x3978df (0xa8e0 bytes) 000073[1] 1744883369.825413 mapwrite 0x820000 thru 0x82429f (0x42a0 bytes) 000074[0] 1744883369.857141 mapread 0x525000 thru 0x528dcf (0x3dd0 bytes) 000075[1] 1744883369.865344 mapread 0x46b000 thru 0x476fdc (0xbfdd bytes) 000076[0] 1744883369.873859 mapwrite 0x653000 thru 0x6560dc (0x30dd bytes) 000077[0] 1744883369.894275 mapread 0x343000 thru 0x34861a (0x561b bytes) 000078[1] 1744883369.901627 mapread 0x4c2000 thru 0x4c75bb (0x55bc bytes) truncating to largest ever: 0x625aa2 000079[1] 1744883369.906628 trunc from 0x8242a0 to 0x625aa1 (0x1fe7fe bytes) 000082[0] 1744883369.915655 mapread 0x60c000 thru 0x612b1a (0x6b1b bytes) 000083[0] 1744883369.923457 mapwrite 0x618000 thru 0x6235cb (0xb5cc bytes) 000084[0] 1744883369.941886 write 0x721000 thru 0x724507 (0x3508 bytes) 000086[0] 1744883369.942663 write 0x0e2000 thru 0x0eeb67 (0xcb68 bytes) 000088[1] 1744883369.947221 write 0x635000 thru 0x6421a5 (0xd1a6 bytes) 000090[1] 1744883369.957662 read 0x0d3000 thru 0x0e02d3 (0xd2d4 bytes) truncating to largest ever: 0x912605 000093[0] 1744883369.962997 trunc from 0x724508 to 0x912604 (0x1ee0fd bytes) 000096[1] 1744883369.976738 read 0x271000 thru 0x274ad1 (0x3ad2 bytes) 000098[1] 1744883369.980890 trunc from 0x912605 to 0x760d1f (0x1b18e5 bytes) 000100[1] 1744883369.986373 write 0x6d0000 thru 0x6d5c45 (0x5c46 bytes) 000101[1] 1744883369.987581 trunc from 0x760d20 to 0x043ba1 (0x71d17e bytes) 000104[0] 1744883369.995437 write 0x570000 thru 0x571e60 (0x1e61 bytes) 000107[0] 1744883370.007104 trunc from 0x571e61 to 0x5b4a87 (0x42c27 bytes) 000109[0] 1744883370.012782 mapread 0x37c000 thru 0x381719 (0x571a bytes) 000113[0] 1744883370.022795 mapread 0x52b000 thru 0x52c520 (0x1521 bytes) 000114[0] 1744883370.028596 mapwrite 0x323000 thru 0x328d77 (0x5d78 bytes) 000117[1] 1744883370.045557 mapread 0x4c3000 thru 0x4c555c (0x255d bytes) 000119[0] 1744883370.053688 write 0x9b5000 thru 0x9c41f7 (0xf1f8 bytes) 000120[1] 1744883370.059673 trunc from 0x9c41f8 to 0x51b138 (0x4a90bf bytes) 000121[0] 1744883370.068800 read 0x28a000 thru 0x29939a (0xf39b bytes) 000125[0] 1744883370.075647 read 0x103000 thru 0x10a5b3 (0x75b4 bytes) 000129[0] 1744883370.079446 read 0x1bf000 thru 0x1c4f09 (0x5f0a bytes) 000130[0] 1744883370.081605 write 0x990000 thru 0x99e2c2 (0xe2c3 bytes) 000131[1] 1744883370.090635 write 0x786000 thru 0x791b71 (0xbb72 bytes) 000133[0] 1744883370.099771 write 0x68e000 thru 0x69a6c1 (0xc6c2 bytes) 000139[1] 1744883370.116554 mapread 0x4e7000 thru 0x4f0d3f (0x9d40 bytes) truncating to largest ever: 0x9775d9 000140[0] 1744883370.126505 trunc from 0x99e2c3 to 0x9775d8 (0x26cea bytes) 000143[0] 1744883370.135972 write 0x370000 thru 0x37a8d1 (0xa8d2 bytes) 000146[0] 1744883370.143709 mapwrite 0x015000 thru 0x0205e2 (0xb5e3 bytes) 000147[0] 1744883370.164982 trunc from 0x9775d9 to 0x7f7f11 (0x17f6c7 bytes) 000149[0] 1744883370.173885 write 0x373000 thru 0x37e864 (0xb865 bytes) 000150[1] 1744883370.178233 read_OD 0x6d3000 thru 0x6d6fff (0x4000 bytes) 000152[0] 1744883370.186230 trunc from 0x7f7f12 to 0x030d24 (0x7c71ed bytes) 000153[0] 1744883370.195909 mapread 0x026000 thru 0x030563 (0xa564 bytes) 000155[0] 1744883370.202793 mapwrite 0x535000 thru 0x53d0fd (0x80fe bytes) 000156[1] 1744883370.229583 mapread 0x3f6000 thru 0x3f6ebf (0xec0 bytes) 000158[0] 1744883370.236577 mapread 0x248000 thru 0x250732 (0x8733 bytes) 000159[1] 1744883370.240375 mapread 0x385000 thru 0x38d907 (0x8908 bytes) 000161[1] 1744883370.243296 mapread 0x1c8000 thru 0x1d6f25 (0xef26 bytes) 000163[1] 1744883370.251000 trunc from 0x53d0fe to 0x7b34ad (0x2763b0 bytes) 000164[0] 1744883370.260436 write 0x4f1000 thru 0x4f31a3 (0x21a4 bytes) 000167[1] 1744883370.270283 write_OD 0x1fe000 thru 0x205fff (0x8000 bytes) 000169[1] 1744883370.281671 read_OD 0x56c000 thru 0x578fff (0xd000 bytes) 000172[1] 1744883370.284617 read_OD 0x315000 thru 0x319fff (0x5000 bytes) 000173[1] 1744883370.286926 mapwrite 0x65a000 thru 0x65f32b (0x532c bytes) 000177[1] 1744883370.313939 mapwrite 0x130000 thru 0x13356e (0x356f bytes) 000178[0] 1744883370.334871 read 0x01a000 thru 0x01e7bf (0x47c0 bytes) 000179[0] 1744883370.340142 mapread 0x554000 thru 0x563df4 (0xfdf5 bytes) 000181[1] 1744883370.348710 trunc from 0x7b34ae to 0x4b81f8 (0x2fb2b5 bytes) 000185[0] 1744883370.364678 mapwrite 0x59e000 thru 0x59ee48 (0xe49 bytes) 000186[0] 1744883370.381175 trunc from 0x59ee49 to 0x4d21ec (0xccc5c bytes) 000189[0] 1744883370.388887 trunc from 0x4d21ed to 0x7f5525 (0x323339 bytes) 000190[0] 1744883370.395583 mapread 0x5f4000 thru 0x603eb1 (0xfeb2 bytes) 000193[0] 1744883370.409033 trunc from 0x7f5526 to 0x782475 (0x730b0 bytes) 000195[0] 1744883370.416334 mapread 0x4e9000 thru 0x4ef1c4 (0x61c5 bytes) 000198[0] 1744883370.434205 read 0x507000 thru 0x509337 (0x2338 bytes) 000199[0] 1744883370.434575 mapread 0x028000 thru 0x035df1 (0xddf2 bytes) 000203[0] 1744883370.456363 read 0x1e7000 thru 0x1f4440 (0xd441 bytes) 000204[1] 1744883370.460249 trunc from 0x782476 to 0x772692 (0xfde3 bytes) 000206[0] 1744883370.475813 mapwrite 0x1a8000 thru 0x1ad84e (0x584f bytes) 000207[0] 1744883370.501478 mapread 0x6f4000 thru 0x6f9022 (0x5023 bytes) 000208[1] 1744883370.509195 trunc from 0x772693 to 0x5b0286 (0x1c240c bytes) 000209[1] 1744883370.516748 write_OD 0x339000 thru 0x345fff (0xd000 bytes) 000215[0] 1744883370.539640 mapwrite 0x55c000 thru 0x55cc29 (0xc2a bytes) 000216[0] 1744883370.555301 mapread 0x16c000 thru 0x17b9c9 (0xf9ca bytes) 000217[1] 1744883370.561705 mapread 0x271000 thru 0x27db9a (0xcb9b bytes) 000219[0] 1744883370.574423 mapwrite 0x1e7000 thru 0x1f477d (0xd77e bytes) 000220[0] 1744883370.601206 write 0x22d000 thru 0x22e20c (0x120d bytes) 000221[1] 1744883370.605424 read_OD 0x172000 thru 0x17dfff (0xc000 bytes) 000226[1] 1744883370.614323 write_OD 0x930000 thru 0x936fff (0x7000 bytes) 000228[0] 1744883370.629374 mapread 0x254000 thru 0x25d30d (0x930e bytes) 000230[0] 1744883370.634313 read 0x010000 thru 0x012028 (0x2029 bytes) 000231[1] 1744883370.635743 trunc from 0x937000 to 0x4a9c60 (0x48d39f bytes) 000232[1] 1744883370.644871 write_OD 0x80c000 thru 0x810fff (0x5000 bytes) 000233[1] 1744883370.661311 trunc from 0x811000 to 0x030b27 (0x7e04d8 bytes) 000234[0] 1744883370.671759 read 0x01f000 thru 0x028091 (0x9092 bytes) 000235[1] 1744883370.678782 mapwrite 0x889000 thru 0x8921ba (0x91bb bytes) 000236[1] 1744883370.705565 read_OD 0x514000 thru 0x51afff (0x7000 bytes) 000237[0] 1744883370.710006 read 0x6b4000 thru 0x6bb334 (0x7335 bytes) 000239[1] 1744883370.712787 read_OD 0x7df000 thru 0x7e2fff (0x4000 bytes) 000240[0] 1744883370.714129 mapread 0x768000 thru 0x76b3a0 (0x33a1 bytes) 000241[0] 1744883370.724438 trunc from 0x8921bb to 0x5038c3 (0x38e8f7 bytes) 000242[1] 1744883370.732121 mapwrite 0x783000 thru 0x78f9bf (0xc9c0 bytes) 000245[0] 1744883370.766897 read 0x76f000 thru 0x776a8e (0x7a8f bytes) 000248[1] 1744883370.771744 read_OD 0x5fc000 thru 0x5fefff (0x3000 bytes) 000249[0] 1744883370.773201 write 0x054000 thru 0x060554 (0xc555 bytes) 000250[0] 1744883370.780199 trunc from 0x78f9c0 to 0x262a43 (0x52cf7c bytes) 000252[1] 1744883370.787733 write_OD 0x5ca000 thru 0x5cdfff (0x4000 bytes) 000254[0] 1744883370.801644 mapread 0x5b4000 thru 0x5c170c (0xd70d bytes) 000258[0] 1744883370.808634 mapwrite 0x398000 thru 0x3a65aa (0xe5ab bytes) 000259[1] 1744883370.833469 mapread 0x1bf000 thru 0x1ce687 (0xf688 bytes) 000261[1] 1744883370.844256 mapread 0x20f000 thru 0x21aa2b (0xba2c bytes) 000263[0] 1744883370.853353 mapwrite 0x91a000 thru 0x92798e (0xd98f bytes) 000265[0] 1744883370.891272 read 0x89b000 thru 0x8a04f2 (0x54f3 bytes) 000266[0] 1744883370.898612 mapread 0x0ed000 thru 0x0fb1b6 (0xe1b7 bytes) 000267[1] 1744883370.907934 write_OD 0x560000 thru 0x563fff (0x4000 bytes) 000272[1] 1744883370.924550 mapread 0x4b9000 thru 0x4c7a64 (0xea65 bytes) 000274[0] 1744883370.931736 mapread 0x912000 thru 0x9209dc (0xe9dd bytes) 000276[1] 1744883370.939579 write_OD 0x563000 thru 0x570fff (0xe000 bytes) 000277[0] 1744883370.953452 read 0x002000 thru 0x009e22 (0x7e23 bytes) 000278[1] 1744883370.957150 mapwrite 0x6eb000 thru 0x6faf46 (0xff47 bytes) 000279[1] 1744883370.981760 mapread 0x01a000 thru 0x023367 (0x9368 bytes) 000281[1] 1744883370.995056 write_OD 0x473000 thru 0x475fff (0x3000 bytes) 000282[0] 1744883371.008244 write 0x86e000 thru 0x87dd98 (0xfd99 bytes) 000283[0] 1744883371.012332 mapread 0x59c000 thru 0x5a9166 (0xd167 bytes) truncating to largest ever: 0x9884cc 000285[1] 1744883371.018679 trunc from 0x92798f to 0x9884cb (0x60b3d bytes) 000286[0] 1744883371.028366 mapwrite 0x9d3000 thru 0x9da106 (0x7107 bytes) 000288[1] 1744883371.058515 read_OD 0x1f4000 thru 0x202fff (0xf000 bytes) 000290[0] 1744883371.065543 trunc from 0x9da107 to 0x51323f (0x4c6ec7 bytes) 000291[0] 1744883371.072725 trunc from 0x513240 to 0x0018e5 (0x51195a bytes) 000292[1] 1744883371.076064 mapwrite 0x1aa000 thru 0x1b7a89 (0xda8a bytes) 000293[0] 1744883371.104324 trunc from 0x1b7a8a to 0x36f2cb (0x1b7842 bytes) 000295[1] 1744883371.115511 read_OD 0x113000 thru 0x11dfff (0xb000 bytes) 000296[0] 1744883371.121793 read 0x344000 thru 0x34d3ea (0x93eb bytes) 000298[0] 1744883371.146376 mapwrite 0x6dc000 thru 0x6dc817 (0x818 bytes) 000301[0] 1744883371.188248 mapwrite 0x12b000 thru 0x135421 (0xa422 bytes) 000303[1] 1744883371.223845 read_OD 0x45f000 thru 0x469fff (0xb000 bytes) 000304[0] 1744883371.234240 read 0x311000 thru 0x314c32 (0x3c33 bytes) 000306[0] 1744883371.237733 write 0x2cf000 thru 0x2dc7b0 (0xd7b1 bytes) 000311[1] 1744883371.250794 read_OD 0x028000 thru 0x029fff (0x2000 bytes) 000314[1] 1744883371.261632 mapwrite 0x19b000 thru 0x1aace5 (0xfce6 bytes) 000315[1] 1744883371.288846 write_OD 0x1a7000 thru 0x1a7fff (0x1000 bytes) 000316[0] 1744883371.300034 mapread 0x219000 thru 0x21f434 (0x6435 bytes) 000318[0] 1744883371.304239 read 0x322000 thru 0x330029 (0xe02a bytes) 000323[0] 1744883371.308349 mapwrite 0x8ef000 thru 0x8f64a5 (0x74a6 bytes) 000327[0] 1744883371.343272 trunc from 0x8f64a6 to 0x2e6b91 (0x60f914 bytes) 000329[0] 1744883371.352509 mapread 0x1c3000 thru 0x1ca32a (0x732b bytes) 000331[0] 1744883371.360123 read 0x197000 thru 0x19a844 (0x3845 bytes) 000333[1] 1744883371.377428 mapread 0x222000 thru 0x231fec (0xffed bytes) 000334[1] 1744883371.387724 mapread 0x1bb000 thru 0x1c1c0d (0x6c0e bytes) 000338[0] 1744883371.390426 mapwrite 0x61a000 thru 0x625d43 (0xbd44 bytes) 000339[0] 1744883371.426370 mapread 0x2c1000 thru 0x2ccaad (0xbaae bytes) 000340[0] 1744883371.438930 mapwrite 0x765000 thru 0x76c4fa (0x74fb bytes) 000343[0] 1744883371.474360 mapwrite 0x51d000 thru 0x520ca7 (0x3ca8 bytes) 000344[1] 1744883371.489143 read_OD 0x008000 thru 0x00afff (0x3000 bytes) 000345[0] 1744883371.497185 write 0x3cc000 thru 0x3db659 (0xf65a bytes) 000347[1] 1744883371.505151 mapread 0x429000 thru 0x42b756 (0x2757 bytes) 000349[1] 1744883371.516840 read_OD 0x14d000 thru 0x155fff (0x9000 bytes) 000350[0] 1744883371.518733 mapwrite 0x0b1000 thru 0x0c0120 (0xf121 bytes) 000353[1] 1744883371.554105 trunc from 0x76c4fb to 0x12ce1c (0x63f6de bytes) 000355[1] 1744883371.562029 trunc from 0x12ce1d to 0x6e0fa7 (0x5b418b bytes) 000356[1] 1744883371.566156 read_OD 0x40e000 thru 0x411fff (0x4000 bytes) 000357[1] 1744883371.570061 mapwrite 0x6eb000 thru 0x6fa326 (0xf327 bytes) 000359[1] 1744883371.597311 write_OD 0x18c000 thru 0x190fff (0x5000 bytes) 000360[0] 1744883371.611105 mapwrite 0x648000 thru 0x64fe03 (0x7e04 bytes) 000361[1] 1744883371.642248 mapread 0x167000 thru 0x169407 (0x2408 bytes) 000362[1] 1744883371.650266 write_OD 0x4fc000 thru 0x506fff (0xb000 bytes) 000363[0] 1744883371.664271 mapwrite 0x41d000 thru 0x424d1a (0x7d1b bytes) 000365[0] 1744883371.695411 read 0x2be000 thru 0x2c2feb (0x4fec bytes) 000371[0] 1744883371.705680 mapread 0x2e2000 thru 0x2f0724 (0xe725 bytes) 000372[1] 1744883371.711539 write_OD 0x9b9000 thru 0x9bcfff (0x4000 bytes) 000373[1] 1744883371.724391 read_OD 0x810000 thru 0x816fff (0x7000 bytes) 000376[0] 1744883371.729622 write 0x42d000 thru 0x42f2a3 (0x22a4 bytes) 000378[0] 1744883371.738907 read 0x3d4000 thru 0x3d8072 (0x4073 bytes) 000379[0] 1744883371.743803 read 0x1bf000 thru 0x1c683c (0x783d bytes) 000382[1] 1744883371.748821 mapwrite 0x155000 thru 0x1618b1 (0xc8b2 bytes) 000386[0] 1744883371.779627 write 0x303000 thru 0x3116cc (0xe6cd bytes) 000388[0] 1744883371.785909 mapwrite 0x6b0000 thru 0x6bfe70 (0xfe71 bytes) 000389[1] 1744883371.804527 read_OD 0x3e9000 thru 0x3f2fff (0xa000 bytes) 000390[0] 1744883371.809913 mapread 0x046000 thru 0x04ac27 (0x4c28 bytes) 000394[0] 1744883371.814450 trunc from 0x9bd000 to 0x5ea4d6 (0x3d2b29 bytes) 000396[1] 1744883371.823475 read_OD 0x064000 thru 0x066fff (0x3000 bytes) 000398[0] 1744883371.828474 read 0x104000 thru 0x10d3e5 (0x93e6 bytes) 000401[0] 1744883371.830548 mapread 0x4f7000 thru 0x4ff5f1 (0x85f2 bytes) 000404[1] 1744883371.834337 mapread 0x179000 thru 0x17b161 (0x2162 bytes) 000408[0] 1744883371.838403 trunc from 0x5ea4d7 to 0x61a472 (0x2ff9c bytes) 000410[0] 1744883371.849791 mapread 0x115000 thru 0x11c51c (0x751d bytes) 000412[1] 1744883371.858179 mapwrite 0x2dc000 thru 0x2e14d9 (0x54da bytes) 000415[1] 1744883371.879955 read_OD 0x169000 thru 0x16ffff (0x7000 bytes) 000420[1] 1744883371.884387 write_OD 0x7f2000 thru 0x7fdfff (0xc000 bytes) 000423[0] 1744883371.900173 write 0x21c000 thru 0x2217db (0x57dc bytes) 000424[0] 1744883371.905911 mapwrite 0x531000 thru 0x53b71f (0xa720 bytes) 000426[0] 1744883371.927382 mapwrite 0x943000 thru 0x945031 (0x2032 bytes) 000430[0] 1744883371.961376 read 0x4ab000 thru 0x4b2c31 (0x7c32 bytes) 000432[0] 1744883371.967787 mapwrite 0x4de000 thru 0x4e1752 (0x3753 bytes) 000434[1] 1744883371.991277 mapwrite 0x78c000 thru 0x797fe7 (0xbfe8 bytes) 000435[1] 1744883372.020674 write_OD 0x451000 thru 0x45bfff (0xb000 bytes) 000437[0] 1744883372.034597 write 0x3c1000 thru 0x3c8dc6 (0x7dc7 bytes) 000441[1] 1744883372.048086 write_OD 0x7c6000 thru 0x7cffff (0xa000 bytes) 000443[0] 1744883372.066969 mapread 0x592000 thru 0x59cafa (0xaafb bytes) 000444[0] 1744883372.074855 read 0x382000 thru 0x385823 (0x3824 bytes) 000447[1] 1744883372.078092 mapread 0x3cf000 thru 0x3dad9b (0xbd9c bytes) 000448[1] 1744883372.084876 read_OD 0x205000 thru 0x20ffff (0xb000 bytes) 000449[0] 1744883372.088284 mapread 0x498000 thru 0x49b4af (0x34b0 bytes) 000452[1] 1744883372.095489 read_OD 0x893000 thru 0x898fff (0x6000 bytes) 000454[1] 1744883372.098692 write_OD 0x1b0000 thru 0x1b2fff (0x3000 bytes) 000455[1] 1744883372.110025 write_OD 0x568000 thru 0x568fff (0x1000 bytes) 000456[0] 1744883372.121984 mapread 0x4f6000 thru 0x504b1b (0xeb1c bytes) 000457[1] 1744883372.128183 write_OD 0x2f9000 thru 0x305fff (0xd000 bytes) 000461[1] 1744883372.142240 mapwrite 0x833000 thru 0x8413b4 (0xe3b5 bytes) 000464[1] 1744883372.163611 mapread 0x687000 thru 0x688962 (0x1963 bytes) 000467[0] 1744883372.169628 mapread 0x498000 thru 0x4a1dd5 (0x9dd6 bytes) 000471[0] 1744883372.177808 mapwrite 0x94f000 thru 0x95e062 (0xf063 bytes) 000473[0] 1744883372.208515 mapwrite 0x292000 thru 0x298e46 (0x6e47 bytes) 000474[0] 1744883372.233179 write 0x29e000 thru 0x2ac7e3 (0xe7e4 bytes) 000477[1] 1744883372.242307 mapread 0x400000 thru 0x4021ff (0x2200 bytes) 000478[1] 1744883372.254946 trunc from 0x95e063 to 0x326709 (0x637959 bytes) 000479[1] 1744883372.269968 mapread 0x30e000 thru 0x30fb8c (0x1b8d bytes) 000481[0] 1744883372.280326 mapwrite 0x569000 thru 0x57697a (0xd97b bytes) 000483[1] 1744883372.323237 read_OD 0x0dd000 thru 0x0e0fff (0x4000 bytes) 000484[1] 1744883372.329054 trunc from 0x57697b to 0x36f8bc (0x2070be bytes) 000485[0] 1744883372.335112 write 0x694000 thru 0x69d975 (0x9976 bytes) 000486[1] 1744883372.340855 trunc from 0x69d976 to 0x775e95 (0xd8520 bytes) 000488[1] 1744883372.356554 mapread 0x5fc000 thru 0x60a891 (0xe892 bytes) 000489[1] 1744883372.371446 read_OD 0x095000 thru 0x098fff (0x4000 bytes) 000490[1] 1744883372.377028 read_OD 0x50e000 thru 0x513fff (0x6000 bytes) 000491[1] 1744883372.380020 read_OD 0x06d000 thru 0x072fff (0x6000 bytes) 000493[1] 1744883372.383019 mapread 0x27d000 thru 0x28bcbf (0xecc0 bytes) 000494[1] 1744883372.391867 trunc from 0x775e96 to 0x283cce (0x4f21c7 bytes) 000499[1] 1744883372.415299 mapread 0x248000 thru 0x24a57f (0x2580 bytes) All operations completed A-OK! PASS 16a (10s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 05:49:33 (1744883373) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.108237 s, 77.5 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.0920695 s, 91.1 MB/s Chance of close/open is 1 in 50 Seed set to 3374 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x9332e1 000002[0] 1744883374.310714 trunc from 00000000 to 0x9332e0 (0x9332e1 bytes) 000004[0] 1744883374.323444 write 0x73d3f8 thru 0x73f293 (0x1e9c bytes) 000005[1] 1744883374.325646 read 0x57762a thru 0x578d31 (0x1708 bytes) 000006[1] 1744883374.330420 trunc from 0x9332e1 to 0x29ecb3 (0x69462d bytes) 000014[1] 1744883374.355127 write 0x886061 thru 0x887e59 (0x1df9 bytes) 000015[1] 1744883374.357493 read 0x4c8d0a thru 0x4ca9ee (0x1ce5 bytes) 000016[1] 1744883374.362593 write 0x4e6c95 thru 0x4e7c0a (0xf76 bytes) 000019[1] 1744883374.376200 read 0x03d5e8 thru 0x03f0c3 (0x1adc bytes) 000020[0] 1744883374.381770 mapwrite 0x72e80d thru 0x72f364 (0xb58 bytes) 000021[0] 1744883374.421527 write 0x29f997 thru 0x2a0077 (0x6e1 bytes) 000023[0] 1744883374.431893 mapread 0x4fabf5 thru 0x4fb900 (0xd0c bytes) 000027[1] 1744883374.437491 mapread 0x739883 thru 0x739f9e (0x71c bytes) 000030[0] 1744883374.447382 trunc from 0x887e5a to 0x1f3c8d (0x6941cc bytes) 000032[1] 1744883374.454063 read 0x122597 thru 0x124035 (0x1a9f bytes) 000033[0] 1744883374.458522 mapread 0x1a71d6 thru 0x1a83fb (0x1226 bytes) 000039[1] 1744883374.475284 trunc from 0x1f3c8e to 0x79171d (0x59da90 bytes) 000043[0] 1744883374.492900 trunc from 0x79171e to 0x378d0b (0x418a12 bytes) 000044[1] 1744883374.504601 write 0x2a32fc thru 0x2a374b (0x450 bytes) 000045[0] 1744883374.511928 trunc from 0x378d0c to 0x8cc21f (0x553514 bytes) 000046[0] 1744883374.522308 mapread 0x2c5160 thru 0x2c660d (0x14ae bytes) 000047[0] 1744883374.527671 read 0x365d9c thru 0x3676b0 (0x1915 bytes) 000048[0] 1744883374.531838 mapread 0x835083 thru 0x835295 (0x213 bytes) 000052[1] 1744883374.537457 trunc from 0x8cc220 to 0x3c8ffc (0x503223 bytes) 000060[0] 1744883374.562901 mapread 0x2563bb thru 0x256532 (0x178 bytes) 000062[1] 1744883374.571877 mapread 0x30f26a thru 0x30f73e (0x4d5 bytes) 000063[0] 1744883374.576028 mapwrite 0x8bf6cf thru 0x8c095b (0x128d bytes) 000064[1] 1744883374.607647 write 0x386d67 thru 0x388240 (0x14da bytes) 000066[0] 1744883374.620595 mapread 0x6945a1 thru 0x6953c3 (0xe23 bytes) 000068[0] 1744883374.631838 write 0x634cec thru 0x636abd (0x1dd2 bytes) 000069[1] 1744883374.637601 read 0x471ec0 thru 0x473bdb (0x1d1c bytes) 000070[0] 1744883374.642591 write 0x0bbfce thru 0x0bc2db (0x30e bytes) 000073[1] 1744883374.653802 mapwrite 0x62a2b7 thru 0x62a4bb (0x205 bytes) 000074[0] 1744883374.674830 mapread 0x818a06 thru 0x81945e (0xa59 bytes) 000080[1] 1744883374.698880 read 0x09aa72 thru 0x09b9ff (0xf8e bytes) 000081[0] 1744883374.704370 trunc from 0x8c095c to 0x613799 (0x2ad1c2 bytes) 000082[0] 1744883374.713992 mapread 0x26b1bf thru 0x26c125 (0xf67 bytes) 000083[0] 1744883374.716586 trunc from 0x61379a to 0x6eafd2 (0xd7839 bytes) 000085[1] 1744883374.726166 mapread 0x1b7198 thru 0x1b8083 (0xeec bytes) 000087[1] 1744883374.744991 mapread 0x4cd9fe thru 0x4ced94 (0x1397 bytes) 000088[1] 1744883374.753412 write 0x06d4f1 thru 0x06d9ea (0x4fa bytes) 000089[1] 1744883374.757112 mapwrite 0x69f973 thru 0x69fe7f (0x50d bytes) 000091[0] 1744883374.774531 mapwrite 0x63f6be thru 0x6407c5 (0x1108 bytes) 000099[1] 1744883374.841670 mapwrite 0x7b198d thru 0x7b2031 (0x6a5 bytes) 000102[1] 1744883374.870555 mapread 0x3a4a54 thru 0x3a6414 (0x19c1 bytes) 000108[1] 1744883374.894252 write 0x8ec62a thru 0x8ed743 (0x111a bytes) 000109[1] 1744883374.895281 trunc from 0x8ed744 to 0x5e0695 (0x30d0ae bytes) 000110[1] 1744883374.906737 mapwrite 0x52b714 thru 0x52c13d (0xa2a bytes) 000111[1] 1744883374.931434 read 0x221cba thru 0x222897 (0xbde bytes) 000112[0] 1744883374.938077 mapwrite 0x7d16a1 thru 0x7d2456 (0xdb6 bytes) 000114[1] 1744883374.980526 trunc from 0x7d2457 to 0x8f2af1 (0x12069b bytes) 000115[0] 1744883374.993293 mapread 0x024cdb thru 0x02506b (0x391 bytes) 000116[0] 1744883375.009982 mapwrite 0x8edd2b thru 0x8efaa7 (0x1d7d bytes) 000117[0] 1744883375.044515 trunc from 0x8f2af2 to 0x3a4dad (0x54dd44 bytes) 000119[0] 1744883375.057844 write 0x65529b thru 0x657005 (0x1d6b bytes) 000122[1] 1744883375.067398 mapwrite 0x183385 thru 0x184e0a (0x1a86 bytes) 000123[1] 1744883375.092217 trunc from 0x657006 to 0x5fcace (0x5a537 bytes) 000125[1] 1744883375.103337 trunc from 0x5fcacf to 0x4b586c (0x147262 bytes) 000129[0] 1744883375.120652 read 0x067db9 thru 0x0694db (0x1723 bytes) 000133[0] 1744883375.137411 read 0x4448e4 thru 0x4458c0 (0xfdd bytes) 000134[1] 1744883375.142580 mapwrite 0x7845ee thru 0x785aaa (0x14bd bytes) 000135[1] 1744883375.168715 write 0x0d1794 thru 0x0d2779 (0xfe6 bytes) 000136[0] 1744883375.174250 trunc from 0x785aab to 0x588eb9 (0x1fcbf1 bytes) 000137[0] 1744883375.189563 mapread 0x09bdcd thru 0x09da9c (0x1cd0 bytes) 000139[0] 1744883375.200672 mapwrite 0x862cf3 thru 0x863c8b (0xf99 bytes) 000140[0] 1744883375.224522 trunc from 0x863c8c to 0x4b14fb (0x3b2790 bytes) 000142[0] 1744883375.233983 read 0x44343a thru 0x444840 (0x1407 bytes) 000145[0] 1744883375.245128 mapread 0x1dc158 thru 0x1dceb4 (0xd5d bytes) 000147[1] 1744883375.251515 read 0x3d3585 thru 0x3d3aa5 (0x521 bytes) 000149[1] 1744883375.256928 mapwrite 0x50f661 thru 0x510a00 (0x13a0 bytes) 000151[0] 1744883375.295342 read 0x2c4f63 thru 0x2c4fae (0x4c bytes) 000153[1] 1744883375.311426 read 0x1854f8 thru 0x187012 (0x1b1b bytes) 000154[1] 1744883375.316464 mapread 0x400b5b thru 0x4028e1 (0x1d87 bytes) 000155[1] 1744883375.320513 write 0x84aa61 thru 0x84b1f6 (0x796 bytes) 000156[0] 1744883375.323746 mapread 0x833c2d thru 0x834b1d (0xef1 bytes) 000157[0] 1744883375.334900 mapread 0x4553f1 thru 0x45563d (0x24d bytes) 000159[1] 1744883375.341625 trunc from 0x84b1f7 to 0x3040e8 (0x54710e bytes) 000167[0] 1744883375.363051 mapwrite 0x955bae thru 0x9569ce (0xe21 bytes) 000169[1] 1744883375.397932 write 0x189290 thru 0x18b0b3 (0x1e24 bytes) 000171[1] 1744883375.409882 write 0x4cf96f thru 0x4d09e3 (0x1075 bytes) 000172[1] 1744883375.419554 trunc from 0x9569cf to 0x5900bf (0x3c690f bytes) 000174[1] 1744883375.424777 mapread 0x37fe05 thru 0x380812 (0xa0e bytes) 000175[1] 1744883375.427885 write 0x0d7fb5 thru 0x0d86c9 (0x715 bytes) 000179[1] 1744883375.435179 read 0x1f8cf1 thru 0x1fa0f6 (0x1406 bytes) 000182[1] 1744883375.440159 write 0x706275 thru 0x707113 (0xe9f bytes) 000186[1] 1744883375.451671 mapwrite 0x241fc4 thru 0x242d36 (0xd73 bytes) 000189[1] 1744883375.483089 write 0x123eea thru 0x123f28 (0x3f bytes) 000190[1] 1744883375.489437 trunc from 0x707114 to 0x5961da (0x170f39 bytes) 000192[1] 1744883375.501156 trunc from 0x5961db to 0x1ed49f (0x3a8d3b bytes) 000193[0] 1744883375.507202 write 0x860bc4 thru 0x8628a5 (0x1ce2 bytes) 000194[0] 1744883375.517309 read 0x839031 thru 0x839ea4 (0xe74 bytes) 000195[1] 1744883375.521739 mapread 0x852e7b thru 0x8547eb (0x1971 bytes) 000196[1] 1744883375.530962 trunc from 0x8628a6 to 0x6d8b0e (0x189d97 bytes) 000197[0] 1744883375.538678 read 0x554d86 thru 0x556aa8 (0x1d23 bytes) 000198[1] 1744883375.542829 write 0x9dfc96 thru 0x9e1507 (0x1872 bytes) 000199[1] 1744883375.543872 mapread 0x45b6f6 thru 0x45bfdd (0x8e8 bytes) 000200[1] 1744883375.550241 trunc from 0x9e1508 to 0x49973f (0x547dc8 bytes) 000201[0] 1744883375.555880 read 0x146f4b thru 0x148b43 (0x1bf9 bytes) 000204[1] 1744883375.569978 mapwrite 0x9f2f4f thru 0x9f3ec0 (0xf72 bytes) 000205[0] 1744883375.595525 trunc from 0x9f3ec1 to 0x4086d5 (0x5eb7eb bytes) 000206[1] 1744883375.607489 write 0x45aea7 thru 0x45be3b (0xf95 bytes) 000207[1] 1744883375.612381 read 0x05cf05 thru 0x05d3c1 (0x4bd bytes) 000208[0] 1744883375.619420 mapwrite 0x9eadfa thru 0x9ecb79 (0x1d80 bytes) 000211[1] 1744883375.650484 mapread 0x6fa83d thru 0x6fc57b (0x1d3f bytes) 000212[0] 1744883375.659653 mapwrite 0x72e4bd thru 0x72f9e1 (0x1525 bytes) 000213[1] 1744883375.676317 trunc from 0x9ecb7a to 0x2f9907 (0x6f3272 bytes) 000214[1] 1744883375.683496 write 0x97eef4 thru 0x980434 (0x1541 bytes) 000215[1] 1744883375.691513 write 0x859d37 thru 0x85aed7 (0x11a1 bytes) 000216[1] 1744883375.694062 mapwrite 0x5dc741 thru 0x5dc937 (0x1f7 bytes) 000217[1] 1744883375.715594 read 0x521a3e thru 0x52311b (0x16de bytes) 000220[1] 1744883375.729078 trunc from 0x980435 to 0x8d60e0 (0xaa354 bytes) 000222[0] 1744883375.740363 mapread 0x5985f6 thru 0x598dc8 (0x7d3 bytes) 000223[1] 1744883375.752448 mapwrite 0x5b84d9 thru 0x5b9ff1 (0x1b19 bytes) 000224[1] 1744883375.775546 mapwrite 0x351eb5 thru 0x3525ce (0x71a bytes) 000229[0] 1744883375.804298 mapread 0x8b5758 thru 0x8b5b6a (0x413 bytes) 000234[1] 1744883375.816803 read 0x6859e9 thru 0x68632d (0x945 bytes) 000238[1] 1744883375.827253 trunc from 0x8d60e1 to 0x4ae9c7 (0x427719 bytes) 000247[0] 1744883375.865432 mapread 0x07503b thru 0x0754e6 (0x4ac bytes) 000248[0] 1744883375.877100 trunc from 0x4ae9c8 to 0x07c391 (0x432636 bytes) 000250[1] 1744883375.891918 mapwrite 0x7415c9 thru 0x741cdd (0x715 bytes) 000251[1] 1744883375.917678 read 0x33c4cb thru 0x33dc3e (0x1774 bytes) 000252[1] 1744883375.922497 read 0x117013 thru 0x118678 (0x1666 bytes) 000253[0] 1744883375.927353 read 0x32c6a8 thru 0x32e46d (0x1dc6 bytes) 000254[1] 1744883375.932392 mapwrite 0x06fa8b thru 0x070749 (0xcbf bytes) 000256[1] 1744883375.954171 trunc from 0x741cde to 0x592f16 (0x1aedc7 bytes) 000262[0] 1744883375.966981 mapread 0x460f10 thru 0x4614c2 (0x5b3 bytes) 000263[0] 1744883375.972588 trunc from 0x592f17 to 0x4be382 (0xd4b94 bytes) 000265[0] 1744883375.983564 mapread 0x106fb6 thru 0x107e13 (0xe5e bytes) 000266[1] 1744883375.989355 mapread 0x2b7929 thru 0x2b811e (0x7f6 bytes) 000267[1] 1744883375.999742 mapread 0x2fa8e0 thru 0x2fadd9 (0x4fa bytes) 000270[1] 1744883376.016222 read 0x1f1fa4 thru 0x1f2dd2 (0xe2f bytes) 000271[1] 1744883376.019853 trunc from 0x4be383 to 0x3859a2 (0x1389e0 bytes) 000272[1] 1744883376.028951 write 0x7d618e thru 0x7d675b (0x5ce bytes) 000274[1] 1744883376.034464 write 0x1e3926 thru 0x1e479d (0xe78 bytes) 000278[0] 1744883376.040472 read 0x2690de thru 0x269880 (0x7a3 bytes) 000281[0] 1744883376.048841 mapwrite 0x8a16a0 thru 0x8a34eb (0x1e4c bytes) 000285[0] 1744883376.085243 read 0x108374 thru 0x10859f (0x22c bytes) 000287[0] 1744883376.090627 mapwrite 0x03094a thru 0x031a62 (0x1119 bytes) 000288[1] 1744883376.111683 read 0x796e91 thru 0x797932 (0xaa2 bytes) 000290[0] 1744883376.123257 write 0x811c6e thru 0x812cb9 (0x104c bytes) 000291[0] 1744883376.127314 write 0x2b8770 thru 0x2b9d06 (0x1597 bytes) 000292[0] 1744883376.129659 read 0x4d25d2 thru 0x4d2f77 (0x9a6 bytes) 000293[0] 1744883376.134923 trunc from 0x8a34ec to 0x545bf6 (0x35d8f5 bytes) 000294[1] 1744883376.143078 read 0x4fcf50 thru 0x4fe162 (0x1213 bytes) 000296[1] 1744883376.147462 read 0x39161a thru 0x3934cb (0x1eb2 bytes) 000300[1] 1744883376.153582 read 0x298c17 thru 0x298e09 (0x1f3 bytes) 000301[0] 1744883376.154948 trunc from 0x545bf7 to 0x6f19fa (0x1abe04 bytes) 000302[0] 1744883376.165250 mapwrite 0x64ae94 thru 0x64b34c (0x4b9 bytes) 000305[1] 1744883376.183996 mapread 0x07423e thru 0x075975 (0x1738 bytes) 000308[1] 1744883376.200848 mapwrite 0x11db32 thru 0x11e7e7 (0xcb6 bytes) 000309[0] 1744883376.226180 trunc from 0x6f19fb to 0x8e2947 (0x1f0f4d bytes) 000310[0] 1744883376.238323 trunc from 0x8e2948 to 0x00852f (0x8da418 bytes) 000311[1] 1744883376.243417 read 0x002904 thru 0x00379c (0xe99 bytes) 000312[1] 1744883376.248159 mapwrite 0x9e6a63 thru 0x9e6c2a (0x1c8 bytes) 000314[0] 1744883376.281143 trunc from 0x9e6c2b to 0x8d57b5 (0x111475 bytes) 000315[1] 1744883376.291428 trunc from 0x8d57b6 to 0x3e2598 (0x4f321d bytes) 000316[1] 1744883376.301016 trunc from 0x3e2599 to 0x57ce9b (0x19a903 bytes) 000318[1] 1744883376.310336 mapwrite 0x16ab40 thru 0x16c62d (0x1aee bytes) 000319[0] 1744883376.334205 mapwrite 0x8af66c thru 0x8b13e1 (0x1d76 bytes) 000320[0] 1744883376.363538 mapread 0x3ce4e8 thru 0x3cedbf (0x8d8 bytes) 000322[1] 1744883376.365996 read 0x759279 thru 0x75a41e (0x11a6 bytes) 000326[1] 1744883376.375170 mapread 0x2a9125 thru 0x2a98ad (0x789 bytes) 000327[0] 1744883376.379535 trunc from 0x8b13e2 to 0x47f15b (0x432286 bytes) 000329[0] 1744883376.390652 write 0x0dfb67 thru 0x0e1b51 (0x1feb bytes) 000332[1] 1744883376.401524 write 0x8b4121 thru 0x8b5347 (0x1227 bytes) 000337[1] 1744883376.423348 read 0x06f5ee thru 0x0710ad (0x1ac0 bytes) 000342[1] 1744883376.445237 read 0x88dbb6 thru 0x88e47b (0x8c6 bytes) 000345[0] 1744883376.457509 mapread 0x77b0b0 thru 0x77ce2a (0x1d7b bytes) 000351[0] 1744883376.480434 mapread 0x26f6e2 thru 0x270a53 (0x1372 bytes) 000354[1] 1744883376.493505 write 0x8a9f0e thru 0x8ab47e (0x1571 bytes) 000357[0] 1744883376.506203 trunc from 0x8b5348 to 0x5cd497 (0x2e7eb0 bytes) 000360[0] 1744883376.522998 read 0x390134 thru 0x391a2f (0x18fc bytes) 000364[1] 1744883376.533594 write 0x9c0d2f thru 0x9c265f (0x1931 bytes) 000367[0] 1744883376.547544 mapwrite 0x8add59 thru 0x8ae431 (0x6d9 bytes) 000372[0] 1744883376.570313 write 0x852b0b thru 0x85466a (0x1b60 bytes) 000374[0] 1744883376.579262 read 0x8ff5e3 thru 0x9015b2 (0x1fd0 bytes) 000375[0] 1744883376.584086 mapread 0x1b28f0 thru 0x1b4214 (0x1925 bytes) 000376[1] 1744883376.593121 mapread 0x23b3cf thru 0x23c1ad (0xddf bytes) 000378[1] 1744883376.607080 trunc from 0x9c2660 to 0x2f1df1 (0x6d086e bytes) 000379[0] 1744883376.617492 trunc from 0x2f1df2 to 0x31f7ee (0x2d9fd bytes) 000382[1] 1744883376.636876 mapwrite 0x029337 thru 0x0297d8 (0x4a2 bytes) 000383[0] 1744883376.659014 read 0x06253e thru 0x06297e (0x441 bytes) 000385[0] 1744883376.684201 write 0x8badd2 thru 0x8bbe3a (0x1069 bytes) 000386[0] 1744883376.698130 read 0x5f2f0d thru 0x5f3c0c (0xd00 bytes) 000387[0] 1744883376.700114 read 0x172fdf thru 0x174333 (0x1355 bytes) 000389[0] 1744883376.705489 mapread 0x7eaf88 thru 0x7eb815 (0x88e bytes) 000390[1] 1744883376.712362 trunc from 0x8bbe3b to 0x46e4f2 (0x44d948 bytes) 000392[0] 1744883376.729988 write 0x2b3bf5 thru 0x2b4cab (0x10b7 bytes) 000393[0] 1744883376.740099 mapwrite 0x0d9360 thru 0x0db2ef (0x1f90 bytes) 000395[1] 1744883376.776758 write 0x002e9e thru 0x00326f (0x3d2 bytes) 000396[0] 1744883376.787307 trunc from 0x46e4f3 to 0x5b1ffb (0x143b09 bytes) 000401[1] 1744883376.804497 write 0x1bb6dd thru 0x1bc4e0 (0xe04 bytes) 000402[1] 1744883376.814331 read 0x2eef20 thru 0x2ef4db (0x5bc bytes) 000403[0] 1744883376.820675 read 0x20e5fb thru 0x20eed9 (0x8df bytes) 000406[0] 1744883376.834600 mapread 0x28b791 thru 0x28cf79 (0x17e9 bytes) 000407[1] 1744883376.837681 read 0x4cf3ee thru 0x4d07aa (0x13bd bytes) 000409[0] 1744883376.845033 mapread 0x34f93f thru 0x350c8f (0x1351 bytes) 000411[1] 1744883376.849235 read 0x10f12b thru 0x110c6e (0x1b44 bytes) 000412[1] 1744883376.851251 trunc from 0x5b1ffc to 0x1255e0 (0x48ca1b bytes) 000415[1] 1744883376.870849 read 0x0e00e0 thru 0x0e079a (0x6bb bytes) 000417[0] 1744883376.881515 write 0x1a73d8 thru 0x1a7deb (0xa14 bytes) 000421[0] 1744883376.901002 mapread 0x0f8cea thru 0x0f9e9f (0x11b6 bytes) 000423[0] 1744883376.913184 mapwrite 0x0bc6a9 thru 0x0bcf4d (0x8a5 bytes) 000425[0] 1744883376.944590 mapread 0x0e3618 thru 0x0e3acb (0x4b4 bytes) 000427[1] 1744883376.953980 mapread 0x1123d3 thru 0x11366e (0x129c bytes) 000428[0] 1744883376.969941 read 0x19dbeb thru 0x19e627 (0xa3d bytes) 000430[1] 1744883376.986932 read 0x0e68f9 thru 0x0e7c02 (0x130a bytes) 000431[1] 1744883377.001198 write 0x2553b0 thru 0x25614e (0xd9f bytes) 000432[1] 1744883377.005474 mapread 0x03a0dd thru 0x03a425 (0x349 bytes) 000434[1] 1744883377.010967 mapwrite 0x4611e7 thru 0x46139b (0x1b5 bytes) 000435[0] 1744883377.047621 mapwrite 0x04440f thru 0x04489a (0x48c bytes) 000436[0] 1744883377.092095 read 0x1c97f7 thru 0x1ca35e (0xb68 bytes) 000437[0] 1744883377.096866 write 0x732a0a thru 0x734569 (0x1b60 bytes) 000439[0] 1744883377.100750 mapread 0x42c788 thru 0x42d45d (0xcd6 bytes) 000440[0] 1744883377.106630 trunc from 0x73456a to 0x5d716e (0x15d3fb bytes) 000445[0] 1744883377.122368 write 0x574dcc thru 0x574e61 (0x96 bytes) 000446[1] 1744883377.127002 write 0x8e4e8c thru 0x8e5931 (0xaa6 bytes) 000448[0] 1744883377.140985 mapread 0x332f21 thru 0x3338c3 (0x9a3 bytes) 000449[0] 1744883377.151967 read 0x03e4ec thru 0x03faaf (0x15c4 bytes) 000450[0] 1744883377.154879 write 0x21eead thru 0x21f8a5 (0x9f9 bytes) 000452[1] 1744883377.167519 read 0x027522 thru 0x02857a (0x1059 bytes) 000453[0] 1744883377.177890 mapread 0x0a81a2 thru 0x0a8ffa (0xe59 bytes) 000454[1] 1744883377.185123 mapread 0x38f08e thru 0x39051c (0x148f bytes) 000458[1] 1744883377.197955 read 0x8bbcb6 thru 0x8bd3ee (0x1739 bytes) 000459[1] 1744883377.203331 write 0x5f9d6d thru 0x5fa28e (0x522 bytes) 000461[1] 1744883377.208607 mapread 0x0ef39c thru 0x0efa64 (0x6c9 bytes) 000462[1] 1744883377.210211 read 0x021893 thru 0x022468 (0xbd6 bytes) 000463[1] 1744883377.211627 mapwrite 0x742780 thru 0x744007 (0x1888 bytes) 000464[0] 1744883377.232288 read 0x36b1c4 thru 0x36c540 (0x137d bytes) 000467[0] 1744883377.237104 read 0x11a618 thru 0x11b17e (0xb67 bytes) 000469[0] 1744883377.241790 write 0x64d4dc thru 0x64f01a (0x1b3f bytes) 000471[1] 1744883377.246798 mapread 0x1ab992 thru 0x1ad3f1 (0x1a60 bytes) 000472[1] 1744883377.257093 mapwrite 0x198426 thru 0x19923a (0xe15 bytes) 000473[0] 1744883377.282406 read 0x052dec thru 0x0543be (0x15d3 bytes) 000476[0] 1744883377.294108 write 0x7af5aa thru 0x7af9d5 (0x42c bytes) 000477[1] 1744883377.296188 mapread 0x881b4e thru 0x881f2f (0x3e2 bytes) 000478[0] 1744883377.303672 mapread 0x060bda thru 0x062121 (0x1548 bytes) 000479[0] 1744883377.316351 mapwrite 0x709b44 thru 0x70a398 (0x855 bytes) 000480[0] 1744883377.343913 trunc from 0x8e5932 to 0x84f936 (0x95ffb bytes) 000481[1] 1744883377.353979 mapwrite 0x9d3173 thru 0x9d3c40 (0xace bytes) 000483[1] 1744883377.393666 mapread 0x304918 thru 0x30563a (0xd23 bytes) 000484[1] 1744883377.398726 read 0x4d7c27 thru 0x4d8c18 (0xff2 bytes) 000485[1] 1744883377.410222 trunc from 0x9d3c41 to 0x0fd29c (0x8d69a4 bytes) 000486[0] 1744883377.418565 trunc from 0x0fd29d to 0x722310 (0x625074 bytes) 000488[0] 1744883377.436076 write 0x58560a thru 0x586d70 (0x1767 bytes) 000489[1] 1744883377.442655 trunc from 0x722311 to 0x6c2dbd (0x5f553 bytes) 000497[0] 1744883377.467506 mapwrite 0x455de9 thru 0x457c42 (0x1e5a bytes) 000498[1] 1744883377.491554 mapread 0x55e622 thru 0x56041e (0x1dfd bytes) All operations completed A-OK! PASS 16b (5s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 05:49:38 (1744883378) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 05:49:39 (1744883379) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.111782 s, 75.0 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.0872287 s, 96.2 MB/s ldlm.namespaces.MGC192.168.203.149@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a9c64000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b5836800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800a9c64000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b5836800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800a9c64000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b5836800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.4902 s, 42.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.703676 s, 149 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.689425 s, 152 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.29871 s, 80.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.75851 s, 59.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.851347 s, 123 MB/s PASS 16d (11s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 05:49:50 (1744883390) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0544333 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309092 s, 33.9 MB/s PASS 16e (1s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 05:49:51 (1744883391) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 05:50:13 (1744883413) TIMEOUT PASS 16g (22s) == sanityn test 16h: mmap read after truncate file ======= 05:50:35 (1744883435) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) == sanityn test 16i: read after truncate file ============ 05:50:37 (1744883437) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (3s) == sanityn test 16j: race dio with buffered i/o ========== 05:50:40 (1744883440) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.374237 s, 70.0 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.331016 s, 79.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.00111 s, 26.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.201584 s, 130 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.968355 s, 27.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.253711 s, 103 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.989092 s, 26.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.433202 s, 60.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.495423 s, 52.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.0201 s, 25.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.09532 s, 23.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.264159 s, 99.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.376284 s, 69.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.16344 s, 22.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.23913 s, 21.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.386209 s, 67.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.525561 s, 49.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.1144 s, 23.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.31359 s, 20.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (12s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 05:50:52 (1744883452) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x32cce75 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Thu Apr 17 05:51:05 EDT 2025 + fsx 2 done: rc=0 Thu Apr 17 05:51:05 EDT 2025 All operations completed A-OK! All operations completed A-OK! + fsx 3 done: rc=0 Thu Apr 17 05:51:05 EDT 2025 All operations completed A-OK! + fsx 4 done: rc=0 Thu Apr 17 05:51:05 EDT 2025 + fsx 5 done: rc=0 Thu Apr 17 05:51:05 EDT 2025 PASS 16k (16s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 05:51:08 (1744883468) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 05:51:12 (1744883472) mmap test1: basic mmap operation (PASS, 0.029648s) mmap test2: MAP_PRIVATE not write back (PASS, 0.018734s) mmap test3: concurrent mmap ops on two nodes (PASS, 4.1075s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.1125s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.050031s) mmap test6: check mmap write/read content on two nodes (PASS, 0.059273s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.33199s) mmap test11: multi-thread shared mmap (PASS, 0.32457s) mmap test10: mtime not change for readonly mmap access (PASS, 1.02s) PASS 18 (13s) == sanityn test 19: test concurrent uncached read races ========================================================================= 05:51:25 (1744883485) oleg349-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg349-client: oleg349-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 05:51:26 (1744883486) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 05:51:28 (1744883488) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) == sanityn test 23: others should see updated atime while another read============================================================== 05:51:29 (1744883489) now is 1744883490 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 new atime is 1744883551 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 05:52:32 (1744883552) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 387 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 387 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 30.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234883 387 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 387 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234883 387 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (1s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 05:52:33 (1744883553) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 05:52:35 (1744883555) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 05:52:36 (1744883556) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 05:52:37 (1744883557) utime: good mknod times 1744883556 <= 1744883557 <= 1744883557 for /mnt/lustre/f26a utime: good mknod times 1744883556 <= 1744883557 <= 1744883557 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 05:52:39 (1744883559) PASS 26b (4s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 05:52:43 (1744883563) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-04-17 05:52:43.000000000 -0400 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-04-17 05:52:43.000000000 -0400 Birth: - PASS 26c (2s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 05:52:46 (1744883566) 0 PASS 30 (3s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 05:52:49 (1744883569) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 05:52:52 (1744883572) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg349-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.149@tcp:/lustre 7542784 38912 7499776 1% /mnt/lustre2 PASS 31b (17s) == sanityn test 31r: open-rename(replace) race =========== 05:53:09 (1744883589) -rw-r--r-- 1 root root 0 Apr 17 05:53 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 05:53:14 (1744883594) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13515 Terminated tail -f $tfile PASS 31s (3s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 05:53:16 (1744883596) PASS 31t (2s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 05:53:18 (1744883598) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 05:53:19 (1744883599) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 05:53:20 (1744883600) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (0s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 05:53:20 (1744883600) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 05:53:21 (1744883601) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c64000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9c64000.ost_server_uuid in IDLE state after 0 sec oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c64000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800a9c64000.ost_server_uuid in FULL state after 0 sec oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c64000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9c64000.ost_server_uuid in IDLE state after 0 sec oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c64000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800a9c64000.ost_server_uuid in FULL state after 0 sec oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c64000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9c64000.ost_server_uuid in IDLE state after 0 sec oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c64000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800a9c64000.ost_server_uuid in FULL state after 0 sec PASS 34 (86s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 05:54:47 (1744883687) fail_loc=0x80000317 Race attempt 0 Wait for 20152 20294 for 60 sec... total: 4000 open/close in 8.42 seconds: 475.31 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.149@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre 192.168.203.149@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 05:55:50 (1744883750) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg349-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.265821 s, 197 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.258242 s, 203 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7497728) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.273147 s, 192 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7493632) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.270424 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7497728) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.255764 s, 205 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7493632) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.302635 s, 173 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7495680) after_dd(7440384) after(7491584) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.296903 s, 177 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7491584) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.279989 s, 187 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7495680) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.268152 s, 196 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.272791 s, 192 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.259542 s, 202 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7497728) after_dd(7442432) after(7493632) PASS 36 (230s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 05:59:40 (1744883980) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 - create 2287 (time 1744883992.39 total 10.00 last 228.59) - create 6554 (time 1744884002.39 total 20.01 last 426.66) total: 10000 create in 27.83 seconds: 359.38 ops/second PASS 37 (32s) == sanityn test 39a: file mtime does not change after rename ========================================================== 06:00:12 (1744884012) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 06:00:14 (1744884014) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 06:00:17 (1744884017) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 06:00:20 (1744884020) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 06:00:22 (1744884022) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 06:00:31 (1744884031) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 06:00:39 (1744884039) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 06:00:47 (1744884047) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 06:00:55 (1744884055) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 06:01:01 (1744884061) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 06:01:06 (1744884066) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 06:01:11 (1744884071) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 06:01:16 (1744884076) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 06:01:21 (1744884081) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 06:01:26 (1744884086) Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 06:01:31 (1744884091) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 06:01:36 (1744884096) Conflict PASS 41h (6s) == sanityn test 41i: reint_open: create vs create ======== 06:01:42 (1744884102) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (681s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 06:13:04 (1744884784) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 06:13:08 (1744884788) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 06:13:12 (1744884792) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 06:13:16 (1744884796) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 06:13:20 (1744884800) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 06:13:24 (1744884804) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 06:13:28 (1744884808) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 06:13:32 (1744884812) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 06:13:36 (1744884816) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 06:13:47 (1744884827) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 06:13:51 (1744884831) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 06:13:55 (1744884835) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 06:13:59 (1744884839) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 06:14:04 (1744884844) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 06:14:08 (1744884848) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 06:14:12 (1744884852) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 06:14:16 (1744884856) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 06:14:17 (1744884857) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (33s) == sanityn test 43k: unlink vs create ==================== 06:14:50 (1744884890) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (367s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 06:20:57 (1744885257) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu Apr 17 06:21:00 EDT 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 06:21:01 (1744885261) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 06:21:06 (1744885266) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 06:21:10 (1744885270) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 06:21:14 (1744885274) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 06:21:19 (1744885279) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 06:21:24 (1744885284) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 06:21:28 (1744885288) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 06:21:32 (1744885292) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 06:21:33 (1744885293) PASS 45a (25s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 06:21:58 (1744885318) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 06:22:02 (1744885322) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 06:22:06 (1744885326) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 06:22:11 (1744885331) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 06:22:16 (1744885336) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 06:22:21 (1744885341) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 06:22:26 (1744885346) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 06:22:30 (1744885350) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 06:22:31 (1744885351) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (388s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 06:28:59 (1744885739) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 06:29:03 (1744885743) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 06:29:07 (1744885747) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 06:29:12 (1744885752) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 06:29:16 (1744885756) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 06:29:20 (1744885760) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 06:29:25 (1744885765) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 06:29:29 (1744885769) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 06:29:33 (1744885773) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 06:29:34 (1744885774) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 06:29:35 (1744885775) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (0s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 06:29:35 (1744885775) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 06:29:36 (1744885776) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 06:29:37 (1744885777) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 06:29:38 (1744885778) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (0s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 06:29:38 (1744885778) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 06:29:39 (1744885779) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.0042477 s, 2.4 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 06:29:46 (1744885786) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00315227 s, 50.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3519: kill: (11195) - No such process PASS 51a (3s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 06:29:49 (1744885789) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00236558 s, 433 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140371 s, 74.7 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 06:30:03 (1744885803) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00570306 s, 180 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 06:30:09 (1744885809) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00757646 s, 138 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 06:30:11 (1744885811) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2455 0x997 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2455 0x997 0x240000400 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 06:30:15 (1744885815) llite.lustre-ffff8800a9c64000.xattr_cache=1 llite.lustre-ffff8800b5836800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:30:15.000000000 -0400 Modify: 2025-04-17 06:30:15.000000000 -0400 Change: 2025-04-17 06:30:15.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:30:21.000000000 -0400 Modify: 2025-04-17 06:30:21.000000000 -0400 Change: 2025-04-17 06:30:21.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:30:26.000000000 -0400 Modify: 2025-04-17 06:30:26.000000000 -0400 Change: 2025-04-17 06:30:26.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:30:26.000000000 -0400 Modify: 2025-04-17 06:30:26.000000000 -0400 Change: 2025-04-17 06:30:26.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:30:32.000000000 -0400 Modify: 2025-04-17 06:30:32.000000000 -0400 Change: 2025-04-17 06:30:32.000000000 -0400 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:30:31.000000000 -0400 Modify: 2025-04-17 06:30:32.000000000 -0400 Change: 2025-04-17 06:30:32.000000000 -0400 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 06:30:38 (1744885838) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 06:30:45 (1744885845) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 06:30:52 (1744885852) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 06:31:04 (1744885864) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 06:31:13 (1744885873) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1744885874 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 3.21258 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (21s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 06:31:34 (1744885894) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1744885896 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.20946 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (33s) == sanityn test 60: Verify data_version behaviour ======== 06:32:08 (1744885928) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 06:32:09 (1744885929) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 06:32:11 (1744885931) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 06:32:13 (1744885933) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00316361 s, 20.7 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 06:32:14 (1744885934) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00623137 s, 10.5 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 06:32:15 (1744885935) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 06:32:16 (1744885936) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00342799 s, 19.1 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 06:32:17 (1744885937) llite.lustre-ffff8800a9c64000.xattr_cache=1 llite.lustre-ffff8800b5836800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 06:32:19 (1744885939) llite.lustre-ffff8800a9c64000.xattr_cache=1 llite.lustre-ffff8800b5836800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 06:32:20 (1744885940) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00202303 s, 506 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00219616 s, 466 kB/s 23618: get lock2 23619: get lock1 23619: done 23619: get lock2 23618: done 23618: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 23618: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 23619: cannot get lock2: Resource deadlock avoided 23619: exit rc=0 23618: done 23618: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 06:32:26 (1744885946) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0131198 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900274 s, 116 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4108: 24260 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 06:32:32 (1744885952) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 06:32:45 (1744885965) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.089092 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101032 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147505 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978678 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187044 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172558 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291946 s, 35.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0343842 s, 30.5 MB/s , 0.0163155 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289918 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170532 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386694 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019631 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318103 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313628 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027948 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356302 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924091 s, 113 MB/s , 0.00933632 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117925 s, 88.9 MB/s , 0.00905761 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014174 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012091 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00442472 s, 237 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135406 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124038 s, 84.5 MB/s 1+0 records in 1+0 records out , 0.012022 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114637 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012412 s, 84.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0132121 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124607 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123433 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015216 s, 68.9 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 06:32:47 (1744885967) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0994937 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768133 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114417 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085696 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129531 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364955 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321809 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426622 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341943 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426356 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345377 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370346 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395358 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420072 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0566951 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05652 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0580465 s, 18.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00760256 s, 138 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105029 s, 99.8 MB/s , 0.00752903 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855118 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112316 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902465 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855142 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103442 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101707 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117437 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011317 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117437 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139365 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119573 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130211 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144442 s, 72.6 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0977745 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103505 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133499 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396634 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329442 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431443 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408262 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0463425 s, 22.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0327249 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460952 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379216 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0517449 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495441 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515092 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492218 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487718 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535889 s, 19.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5836800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5836800.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748156 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922146 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0085095 s, 123 MB/s , 0.00875324 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961159 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124335 s, 84.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0045545 s, 230 MB/s , 0.00906042 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144376 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097337 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00750186 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108573 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123304 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948109 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00988816 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102259 s, 103 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 06:32:51 (1744885971) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0881527 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898085 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101606 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321596 s, 32.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0452149 s, 23.2 MB/s , 0.0453062 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037238 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459434 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495807 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500456 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508563 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528066 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481495 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515011 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565368 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0718642 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943355 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127871 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105627 s, 99.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127173 s, 82.5 MB/s , 0.0111459 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130395 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127448 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120504 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134094 s, 78.2 MB/s , 0.0123226 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134766 s, 77.8 MB/s , 0.0127982 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013037 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153216 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144209 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171922 s, 61.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0917635 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118381 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140357 s, 74.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430741 s, 24.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0366454 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448133 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466301 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372361 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051347 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047815 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563367 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0692041 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0654644 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0712476 s, 14.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0699765 s, 15.0 MB/s 1048576 bytes (1.0 MB) copied, 0.068258 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0732791 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140891 s, 74.4 MB/s , 0.00892307 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00729444 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081573 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147985 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106674 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099131 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962402 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100801 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177963 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186109 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132062 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120268 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127782 s, 82.1 MB/s , 0.014774 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162645 s, 64.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0972687 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104613 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119917 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105518 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312153 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036685 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410217 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433051 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471207 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390022 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040683 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458359 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440046 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0522959 s, 20.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0450781 s, 23.3 MB/s , 0.0556585 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0558132 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00722566 s, 145 MB/s , 0.00923954 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104527 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00796067 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800039 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817497 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875068 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010582 s, 99.1 MB/s , 0.010408 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103685 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102838 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119516 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114777 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106235 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116868 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115895 s, 90.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (3s) == sanityn test 77d: check TRR nrs policy ================ 06:32:54 (1744885974) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.103308 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100494 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128257 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259483 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306665 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317607 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341258 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0389771 s, 26.9 MB/s , 0.0390337 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347679 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464519 s, 22.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0484359 s, 21.6 MB/s , 0.045804 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447351 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501189 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512811 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554747 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00844594 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.00549849 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167512 s, 62.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800476 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812615 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198793 s, 52.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0100635 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.00801088 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909715 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763823 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920499 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944208 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916512 s, 114 MB/s , 0.0231985 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010318 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123654 s, 84.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0930862 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995703 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119075 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334858 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350646 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355926 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330947 s, 31.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371259 s, 28.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0363648 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373738 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0629717 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0683193 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.068653 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0699045 s, 15.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0735031 s, 14.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0773315 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0795947 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00708303 s, 148 MB/s 1048576 bytes (1.0 MB) copied, 0.00917331 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769319 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995074 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906296 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897348 s, 117 MB/s , 0.00852121 s, 123 MB/s 1048576 bytes (1.0 MB) copied, 0.00499164 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110623 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100045 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959944 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983083 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010136 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103166 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123916 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153136 s, 68.5 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0916392 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109689 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120834 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127499 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014524 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360773 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0465629 s, 22.5 MB/s , 0.05265 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473865 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0547478 s, 19.2 MB/s 1048576 bytes (1.0 MB) copied, 0.053072 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512942 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386737 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457491 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500435 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569947 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544428 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.006347 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013625 s, 77.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00681939 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701766 s, 149 MB/s , 0.0126528 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813452 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856596 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883964 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103786 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103055 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113017 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113336 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110942 s, 94.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00520295 s, 202 MB/s 1048576 bytes (1.0 MB) copied, 0.0124419 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149367 s, 70.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 06:32:58 (1744885978) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.49@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.112988 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826895 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114644 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170589 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104451 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120455 s, 87.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0397457 s, 26.4 MB/s , 0.0353914 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402531 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452971 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466598 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466359 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453044 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172173 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175115 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172182 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.194434 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939054 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833643 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847968 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128125 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941985 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130882 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133004 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115752 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011797 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011516 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129389 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129062 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0128243 s, 81.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0136542 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133627 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139358 s, 75.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0927125 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699464 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118047 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139437 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150345 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192199 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262205 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348541 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269856 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269841 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421533 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030515 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0560346 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518165 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0644024 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055904 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0718768 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00757044 s, 139 MB/s , 0.00984383 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941511 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117253 s, 89.4 MB/s 1048576 bytes (1.0 MB) copied, 0.00917423 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871312 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991919 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979388 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105126 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107261 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111418 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122366 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119362 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135037 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011251 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012448 s, 84.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0863809 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998195 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163256 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132028 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143061 s, 73.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350451 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023211 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388252 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247044 s, 42.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0367226 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038183 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457276 s, 22.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0453331 s, 23.1 MB/s , 0.0466413 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409699 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408292 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465892 s, 22.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974807 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00733245 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849145 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961048 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123403 s, 85.0 MB/s 1048576 bytes (1.0 MB) copied, 0.00950209 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104556 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0101364 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0111053 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095524 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104954 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011127 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113885 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010894 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119402 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111992 s, 93.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.100612 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122504 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129853 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037759 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436153 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379152 s, 27.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436725 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394617 s, 26.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0490634 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377446 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442624 s, 23.7 MB/s , 0.0556218 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0548629 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057687 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471764 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0574746 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0610782 s, 17.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738031 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884239 s, 119 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092024 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931148 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.0209209 s, 50.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0204159 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100379 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192111 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104485 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226658 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.011921 s, 88.0 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0123918 s, 84.6 MB/s , 0.0123828 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113395 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130036 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138939 s, 75.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 06:33:04 (1744885984) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.115843 s, 145 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974335 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119078 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116307 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125976 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441275 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365219 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495449 s, 21.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.036962 s, 28.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0386233 s, 27.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0503344 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518668 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397913 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532132 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429366 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0543728 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589391 s, 17.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00780808 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007635 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861864 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085326 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114606 s, 91.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125518 s, 83.5 MB/s , 0.00463723 s, 226 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890661 s, 118 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958281 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128635 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946679 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.013382 s, 78.4 MB/s , 0.0112578 s, 93.1 MB/s , 0.00151926 s, 690 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115694 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148045 s, 70.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.153455 s, 109 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926943 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163276 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371728 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292735 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027202 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401968 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433123 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505401 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538459 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457258 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443507 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512872 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476441 s, 22.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470994 s, 22.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0605677 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.062126 s, 16.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954422 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926031 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00762643 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708069 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0084733 s, 124 MB/s , 0.00962632 s, 109 MB/s , 0.00910469 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156324 s, 671 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932125 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0131041 s, 80.0 MB/s 1048576 bytes (1.0 MB) copied, 0.00189616 s, 553 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993317 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840416 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155493 s, 674 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114419 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119336 s, 87.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0946173 s, 177 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117469 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112053 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288207 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311321 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345825 s, 30.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366201 s, 28.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0344493 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375684 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432542 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402394 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427325 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372002 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408818 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398048 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434873 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411831 s, 25.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00941019 s, 111 MB/s , 0.00736904 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874268 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102911 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933847 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966226 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117553 s, 89.2 MB/s , 0.0115083 s, 91.1 MB/s , 0.0113332 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115074 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0129399 s, 81.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0112666 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102267 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116264 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118176 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137525 s, 76.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0972594 s, 172 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897243 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127361 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326355 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345996 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366506 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270454 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364449 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403832 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379476 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308105 s, 34.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387436 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423652 s, 24.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0460885 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387074 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427962 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0717744 s, 14.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00455845 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00496348 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101805 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748338 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00655135 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977366 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791753 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00792406 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779691 s, 134 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00178212 s, 588 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00146529 s, 716 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112114 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106586 s, 98.4 MB/s , 0.00319112 s, 329 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00175683 s, 597 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0014437 s, 726 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 06:33:10 (1744885990) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 06:33:13 (1744885993) oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg349-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg349-client: oleg349-server: ssh exited with exit code 2 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg349-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg349-client: oleg349-server: ssh exited with exit code 12 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg349-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg349-client: oleg349-server: ssh exited with exit code 12 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg349-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg349-client: oleg349-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 06:33:16 (1744885996) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 06:33:22 (1744886002) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0786932 s, 213 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092765 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136394 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294627 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295785 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315419 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315367 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300967 s, 34.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247671 s, 42.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0281502 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345941 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294868 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290292 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341564 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0323975 s, 32.4 MB/s , 0.031377 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417179 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103089 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00752279 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972937 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00827277 s, 127 MB/s , 0.00827007 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916217 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102515 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.010421 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.00541993 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100455 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103154 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117448 s, 89.3 MB/s , 0.0112213 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109748 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121631 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167909 s, 62.4 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93602 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4376 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 06:34:11 (1744886051) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.441516 s, 38.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192195 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173959 s, 60.3 MB/s , 0.0175089 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017831 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175855 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306412 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298071 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184414 s, 56.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.148362 s, 7.1 MB/s , 0.14753 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.158169 s, 6.6 MB/s , 0.160056 s, 6.6 MB/s 1048576 bytes (1.0 MB) copied, 0.161787 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15938 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150925 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169729 s, 6.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944627 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012046 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0111617 s, 93.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0113069 s, 92.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134629 s, 77.9 MB/s , 0.0116245 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111839 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108028 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160743 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121433 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163405 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139221 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0132722 s, 79.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0134202 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138845 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153031 s, 68.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.829 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9673 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.44623 s, 37.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113164 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118722 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142113 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011048 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016104 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864904 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723921 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779942 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145012 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158121 s, 6.6 MB/s 1048576 bytes (1.0 MB) copied, 0.136207 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156912 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148753 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143253 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167371 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154138 s, 6.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00890809 s, 118 MB/s , 0.00920447 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954466 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109582 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123951 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984541 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101795 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137197 s, 76.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130964 s, 80.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00646127 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113574 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123239 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013246 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131198 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123251 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138275 s, 75.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0299 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9602 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (130s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 06:36:21 (1744886181) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.21761 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176207 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261294 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026521 s, 39.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260558 s, 40.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168212 s, 62.3 MB/s , 0.0277419 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100023 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188808 s, 55.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.176916 s, 5.9 MB/s , 0.176888 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.170414 s, 6.2 MB/s , 0.188584 s, 5.6 MB/s , 0.178143 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187079 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18574 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188958 s, 5.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875203 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673925 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980599 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104437 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00838807 s, 125 MB/s , 0.00855496 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113737 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122711 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010687 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107316 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980587 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104879 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115457 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106293 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117755 s, 89.0 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0281 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9625 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg349-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.449782 s, 37.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152492 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167769 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222825 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217503 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270157 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292947 s, 35.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0344136 s, 30.5 MB/s , 0.029833 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.138586 s, 7.6 MB/s 1048576 bytes (1.0 MB) copied, 0.141404 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.133985 s, 7.8 MB/s , 0.126134 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151487 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.151583 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13484 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13157 s, 8.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106071 s, 98.9 MB/s , 0.0103312 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092342 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113012 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116223 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125677 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00387972 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109879 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102351 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00971692 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0114324 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957561 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105327 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110293 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980674 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114884 s, 91.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0272 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9629 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (130s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 06:38:31 (1744886311) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.250322 s, 67.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779993 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268033 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289583 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241499 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306313 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156221 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335282 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236598 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385495 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392363 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220912 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027372 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261291 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274438 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413869 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496999 s, 21.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708378 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00694144 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00715876 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00682317 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113925 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108544 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101261 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811642 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861064 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00604762 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890785 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964971 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00932589 s, 112 MB/s , 0.00171057 s, 613 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165967 s, 632 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0016887 s, 621 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94791 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73021 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.49@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.49@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0980128 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081862 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131936 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288462 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328233 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308452 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322379 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358074 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247998 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323259 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328697 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378196 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034797 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.218118 s, 4.8 MB/s , 0.215185 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.261371 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.267359 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932641 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851836 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00146124 s, 718 MB/s , 0.00865123 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00504622 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012866 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110113 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144565 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116665 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125863 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120426 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122333 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127732 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124085 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120311 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141912 s, 73.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9281 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72997 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.49@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107652 s, 156 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010219 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109571 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0125837 s, 83.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0270564 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278501 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219899 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218824 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303836 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378539 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293552 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371756 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397105 s, 26.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562875 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536408 s, 19.5 MB/s 1048576 bytes (1.0 MB) copied, 0.057156 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0581069 s, 18.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00693526 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819234 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814372 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805699 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103785 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00119725 s, 876 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090796 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910994 s, 115 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00923613 s, 114 MB/s , 0.00983815 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.00915917 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105963 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108589 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904436 s, 116 MB/s , 0.0101226 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011646 s, 90.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93562 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87953 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.101116 s, 166 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906834 s, 116 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246991 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01918 s, 54.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0292247 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0178106 s, 58.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0304298 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186607 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296263 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331733 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512771 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441971 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461421 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122118 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120415 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111138 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115231 s, 9.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00752666 s, 139 MB/s , 0.0087409 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111689 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684398 s, 153 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101482 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0105225 s, 99.7 MB/s , 0.0118429 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901203 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105204 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113718 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0113414 s, 92.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0120457 s, 87.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0136839 s, 76.6 MB/s , 0.0119898 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121563 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129873 s, 80.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78582 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73037 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9073 s, 21.4 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72761 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0996902 s, 168 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759249 s, 138 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194289 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198957 s, 52.7 MB/s 1048576 bytes (1.0 MB) copied, 0.017336 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264141 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283802 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270385 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225255 s, 46.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1099 s, 9.5 MB/s 1048576 bytes (1.0 MB) copied, 0.114961 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118329 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125944 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118882 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109153 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125829 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128861 s, 8.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00954793 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.0094896 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00795992 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836786 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114058 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103768 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953953 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865951 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771389 s, 136 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856117 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106324 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894685 s, 117 MB/s 1048576 bytes (1.0 MB) copied, 0.00995536 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.0101998 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983056 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129666 s, 80.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8268 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4273 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0996589 s, 168 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877334 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100929 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923716 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081697 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191636 s, 54.7 MB/s , 0.0215364 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107718 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216047 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141465 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355547 s, 29.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275653 s, 38.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0161368 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444483 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312959 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436251 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283246 s, 37.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839344 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893507 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011194 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846698 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111108 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102947 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102419 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995624 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115482 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086876 s, 121 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0115684 s, 90.6 MB/s , 0.00878949 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103515 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124922 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102457 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124317 s, 84.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9294 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72909 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0919886 s, 182 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834571 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149463 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150875 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231317 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243407 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134917 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034726 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352497 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330863 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355495 s, 29.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.127854 s, 8.2 MB/s , 0.137584 s, 7.6 MB/s 1048576 bytes (1.0 MB) copied, 0.134218 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121931 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.269779 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.274981 s, 3.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849593 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00523602 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135807 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126762 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104884 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102438 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864276 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977224 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132073 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011353 s, 92.4 MB/s , 0.0114421 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150201 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129852 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133235 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00240821 s, 435 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149799 s, 70.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77525 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73371 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76125 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72963 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (351s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 06:44:22 (1744886662) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 06:44:25 (1744886665) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg349-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.71316 s, 282 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg349-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0167 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 06:45:15 (1744886715) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0839942 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811013 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117271 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168078 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279821 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208034 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243751 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282654 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370841 s, 28.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419274 s, 25.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0356596 s, 29.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0381797 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451552 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400878 s, 26.2 MB/s , 0.041883 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405228 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529237 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979264 s, 107 MB/s , 0.00849729 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859303 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00705234 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100391 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899931 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00858012 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956754 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100434 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108238 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995799 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105644 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107123 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119182 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117099 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135373 s, 77.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9182 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96674 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.229944 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984346 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225179 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205978 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274481 s, 38.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303743 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283855 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304679 s, 34.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0335689 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379242 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326218 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384505 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366388 s, 28.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.271128 s, 3.9 MB/s , 0.274535 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.317597 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.323913 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236193 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209369 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233922 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218504 s, 48.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0222182 s, 47.2 MB/s , 0.0221494 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022276 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226512 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235171 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229464 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231512 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230801 s, 45.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239495 s, 43.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0251313 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236742 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249568 s, 42.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg349-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg349-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91697 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96739 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (62s) == sanityn test 77o: Changing rank should not panic ====== 06:46:17 (1744886777) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 06:46:20 (1744886780) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (25s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 06:46:45 (1744886805) mds.MDS.mdt.nrs_policies=tbf oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg349-client: oleg349-server: ssh exited with exit code 36 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg349-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg349-client: oleg349-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 06:46:55 (1744886815) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 21717 kill md_thread xargs PASS 77r (38s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 06:47:32 (1744886852) oleg349-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg349-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg349-client: oleg349-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 06:47:34 (1744886854) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506690 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:47:34.000000000 -0400 Modify: 2025-04-17 06:47:34.000000000 -0400 Change: 2025-04-17 06:47:34.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 06:47:46 (1744886866) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 06:47:47 (1744886867) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 06:47:48 (1744886868) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 06:47:49 (1744886869) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 06:47:50 (1744886870) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 06:47:51 (1744886871) total: 2500 open/close in 5.11 seconds: 489.25 ops/second snapshot_time 1744886890.854907037 secs.nsecs start_time 1744886871.786151195 secs.nsecs elapsed_time 19.068755842 secs.nsecs open 2505 samples [usecs] 51 6925 668990 243934758 close 2504 samples [usecs] 47 2635 207742 25214666 mknod 2501 samples [usecs] 149 6109 586490 188431510 unlink 2500 samples [usecs] 250 14551 1124403 1751631553 mkdir 2 samples [usecs] 455 701 1156 698426 rmdir 2 samples [usecs] 2196 2748 4944 12373920 rename 2500 samples [usecs] 212 14445 2424830 4009638568 getattr 3760 samples [usecs] 5 87 53114 981840 parallel_rename_file 2500 samples [usecs] 212 14445 2424830 4009638568 crossdir_rename 2500 samples [usecs] 212 14445 2424830 4009638568 PASS 81d (21s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 06:48:12 (1744886892) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7038 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7038 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 06:48:15 (1744886895) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 06:48:16 (1744886896) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 85: Lustre API root cache race =========== 06:48:23 (1744886903) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1744886903 Finishing test test1 at 1744886904 PASS 85 (2s) == sanityn test 90: open/create and unlink striped directory ========================================================== 06:48:25 (1744886905) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 06:48:26 (1744886906) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 06:48:27 (1744886907) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 06:48:28 (1744886908) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5997 0x176d 0x240000400 1 5982 0x175e 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5983 0x175f 0x280000400 0 5998 0x176e 0x240000400 rc1=0 and rc2=0 PASS 93 (7s) == sanityn test 94: signal vs CP callback race =========== 06:48:35 (1744886915) osc.lustre-OST0000-osc-ffff8800a9c64000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b5836800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800a9c64000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b5836800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.034004 s, 241 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99952 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5764: 10367 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00944164 s, 434 kB/s osc.lustre-OST0000-osc-ffff8800a9c64000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b5836800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9c64000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5836800.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 06:48:45 (1744886925) llite.lustre-ffff8800a9c64000.fast_read=0 llite.lustre-ffff8800b5836800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0120799 s, 174 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800a9c64000.fast_read=1 llite.lustre-ffff8800b5836800.fast_read=1 PASS 95a (19s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 06:49:04 (1744886944) llite.lustre-ffff8800a9c64000.fast_read=0 llite.lustre-ffff8800b5836800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.0014395 s, 8.5 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800a9c64000.fast_read=1 llite.lustre-ffff8800b5836800.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 06:49:11 (1744886951) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 06:49:12 (1744886952) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00149545 s, 2.7 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 06:49:14 (1744886954) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0187007 s, 112 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 06:49:15 (1744886955) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.019129 s, 110 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 06:49:17 (1744886957) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00148819 s, 8.1 MB/s PASS 100e (1s) == sanityn test 101a: Discard DoM data on unlink ========= 06:49:18 (1744886958) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000465219 s, 8.8 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 06:49:20 (1744886960) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000809655 s, 5.1 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 06:49:22 (1744886962) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000490278 s, 8.4 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 06:49:25 (1744886965) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1744886965 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1744886968 Finishing test test23 at 1744886968 PASS 103 (6s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 06:49:31 (1744886971) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 06:49:32 (1744886972) fail_loc=0x416 PASS 105 (27s) == sanityn test 106a: Verify the btime via statx() ======= 06:49:59 (1744886999) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 06:50:00 (1744887000) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 06:50:02 (1744887002) SKIP: sanityn test_106c statx() only test SKIP 106c (2s) == sanityn test 107a: Basic grouplock conflict =========== 06:50:03 (1744887003) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.071541 s, 147 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6005 0x1775 0x240000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 06:50:07 (1744887007) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0663052 s, 158 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5989 0x1765 0x280000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7038 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 06:50:15 (1744887015) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00166845 s, 6.0 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.21613 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 06:50:21 (1744887021) 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 3 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 12 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg349-server: libkmod: kmod_module_get_size: failed to read coresize from /sys/module/intel_rapl Iteration 13 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 17 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 26 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 27 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 28 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 32 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 34 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 35 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 36 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 42 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg349-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 50 Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 192.168.203.149@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.149@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg349-server' oleg349-server: oleg349-server.virtnet: executing load_modules_local oleg349-server: Loading modules from /home/green/git/lustre-release/lustre oleg349-server: detected 4 online CPUs by sysfs oleg349-server: MODOPTS_LIBCFS= oleg349-server: Force libcfs to create 2 CPU partitions Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre Starting client: oleg349-client.virtnet: -o user_xattr,flock 192.168.203.149@tcp:/lustre /mnt/lustre2 PASS 109 (433s) == sanityn test 110: do not grant another lock on resend ========================================================== 06:57:34 (1744887454) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-17 06:57:35.000000000 -0400 Modify: 2025-04-17 06:57:35.000000000 -0400 Change: 2025-04-17 06:57:35.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg349-client.virtnet oleg349-client.virtnet PASS 110 (65s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 06:58:39 (1744887519) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 06:58:40 (1744887520) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 06:58:41 (1744887521) lustre-OST0000-osc-ffff88012a5d9800 active. lustre-OST0001-osc-ffff88012a5d9800 active. lustre-OST0000-osc-ffff88012a5d9800 active. lustre-OST0001-osc-ffff88012a5d9800 active. MGC192.168.203.149@tcp active. PASS 113 (3s) == sanityn test 114: implicit default LMV inherit ======== 06:58:44 (1744887524) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 06:58:45 (1744887525) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 06:58:46 (1744887526) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 06:58:47 (1744887527) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=20 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1186 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 20.1308 s, 0.2 kB/s , 20.1239 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1339 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1277 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1262 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1219 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.13 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.124 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.12 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1389 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2244 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2388 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 40.2373 s, 0.1 kB/s , 40.2341 s, 0.1 kB/s , 40.2122 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2237 s, 0.1 kB/s , 40.2335 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2356 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.2485 s, 0.1 kB/s , 60.2616 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.2604 s, 0.1 kB/s , 60.2572 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2405 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3243 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.3317 s, 0.1 kB/s , 60.3569 s, 0.1 kB/s , 60.3143 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.244 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.3665 s, 0.1 kB/s , 80.351 s, 0.1 kB/s , 80.3544 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3529 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.431 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.4405 s, 0.1 kB/s , 80.4376 s, 0.1 kB/s , 80.4505 s, 0.1 kB/s , 80.4392 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4748 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.389 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg349-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9cbb000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9cbb000.ost_server_uuid in IDLE state after 0 sec PASS 200 (107s) cleanup: ====================================================== == sanityn test complete, duration 4485 sec ============== 07:00:34 (1744887634) === sanityn: start cleanup 07:00:35 (1744887635) === Stopping clients: oleg349-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg349-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 07:01:34 (1744887694) ===