-----============= acceptance-small: sanityn ============----- Thu May 1 17:42:03 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg121-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 17:42:20 (1746135740) === Starting client oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Started clients oleg121-client.virtnet: 192.168.201.121@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg121-client.virtnet: executing check_config_client /mnt/lustre oleg121-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg121-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa034800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800af65b000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa034800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800af65b000.idle_timeout=debug disable quota as required oleg121-server: oleg121-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 17:42:40 (1746135760) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480508 s, 21.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7063] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 17:42:42 (1746135762) PASS 0 (5s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 17:42:47 (1746135767) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (5s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 17:42:52 (1746135772) -rw-r--r-- 1 root root 0 May 1 17:42 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 17:42:56 (1746135776) -rw-r--r-- 1 root root 0 May 1 17:42 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 17:43:00 (1746135780) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 17:43:04 (1746135784) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 17:43:09 (1746135789) total 1 -rwxrwxrwx 1 root root 0 May 1 17:42 f2a -rwxrwxrwx 1 root root 0 May 1 17:42 f2b total 1 -rwxrwxrwx 1 root root 0 May 1 17:42 f2a -rwxrwxrwx 1 root root 0 May 1 17:42 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.0134337 s, 38.1 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 17:43:13 (1746135793) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 17:43:15 (1746135795) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.336078 s, 6.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (5s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 17:43:21 (1746135801) PASS 3 (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 17:43:25 (1746135805) PASS 4 (6s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 17:43:31 (1746135811) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (5s) == sanityn test 6: remove of open file on other node ============================================================================ 17:43:36 (1746135816) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (5s) == sanityn test 7: remove of open directory on other node ======================================================================= 17:43:41 (1746135821) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (5s) == sanityn test 8: remove of open special file on other node ==================================================================== 17:43:46 (1746135826) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (7s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 17:43:53 (1746135833) PASS 9a (6s) == sanityn test 9b: append to striped sparse file ======== 17:43:59 (1746135839) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452258 s, 23.2 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0369065 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (5s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 17:44:05 (1746135845) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0132088 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0238138 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00813526 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00691291 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00949851 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0100218 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00608858 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0271248 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0134282 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0158267 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0171136 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0063747 s, 0.2 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 17:44:11 (1746135851) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0129436 s, 237 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.022664 s, 181 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000939578 s, 3.3 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 17:44:15 (1746135855) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (5s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 17:44:20 (1746135860) warning: '-runtime' deprecated, use '-t runtime' instead - link 463 (time 1746135872.31 total 10.02 last 46.21) - link 939 (time 1746135882.31 total 20.03 last 47.57) - link 1373 (time 1746135892.32 total 30.04 last 43.36) - link 1820 (time 1746135902.33 total 40.04 last 44.69) - link 2253 (time 1746135912.34 total 50.05 last 43.26) - link 2699 (time 1746135922.34 total 60.06 last 44.58) - link 3221 (time 1746135932.35 total 70.06 last 52.17) - link 3693 (time 1746135942.36 total 80.08 last 47.14) - link 4251 (time 1746135952.38 total 90.09 last 55.71) total: 4752 link in 100.00 seconds: 47.52 ops/second using seed 3448129662 running for 100 seconds - stat 9018 (time 1746135874 ; total 11 ; last 11) - stat 10000 (time 1746135875 ; total 12 ; last 1) - stat 19699 (time 1746135886 ; total 23 ; last 11) - stat 20000 (time 1746135886 ; total 23 ; last 0) - stat 28697 (time 1746135897 ; total 34 ; last 11) - stat 30000 (time 1746135898 ; total 35 ; last 1) - stat 38861 (time 1746135909 ; total 46 ; last 11) - stat 40000 (time 1746135910 ; total 47 ; last 1) - stat 43664 (time 1746135921 ; total 58 ; last 11) - stat 45625 (time 1746135932 ; total 69 ; last 11) - stat 47476 (time 1746135943 ; total 80 ; last 11) - stat 49420 (time 1746135954 ; total 91 ; last 11) - stat 50000 (time 1746135958 ; total 95 ; last 4) total: 50831 stats in 100 seconds: 508.309998 stats/second - unlinked 0 (time 1746135913 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile4752) error: No such file or directory total: 4751 unlinks in 85 seconds: 55.894119 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22144) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22147) - No such process PASS 12 (142s) == sanityn test 13: test directory page revocation ======= 17:46:42 (1746136002) f13 f13 PASS 13 (6s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 17:46:48 (1746136008) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 23542 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (6s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 17:46:54 (1746136014) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 24179 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (5s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 17:46:59 (1746136019) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24814 Terminated $DIR1/$tdir/sleep 60 PASS 14b (5s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 17:47:05 (1746136025) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25454 Terminated $DIR1/$tdir/sleep 60 PASS 14c (6s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 17:47:10 (1746136030) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 26092 Terminated $DIR1/$tdir/sleep 60 PASS 14d (8s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 17:47:18 (1746136038) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (22s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 17:47:40 (1746136060) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.505756 s, 16.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.400993 s, 20.9 MB/s Chance of close/open is 1 in 50 Seed set to 6063 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1746136064.115812 write 0x7a096b thru 0x7abbcc (0xb262 bytes) 000003[1] 1746136064.168882 read 0x41f5a4 thru 0x42e3df (0xee3c bytes) 000004[0] 1746136064.246463 write 0x016110 thru 0x0242f3 (0xe1e4 bytes) truncating to largest ever: 0x794773 000006[0] 1746136064.292566 trunc from 0x7abbcd to 0x794772 (0x1745a bytes) 000012[1] 1746136064.402842 read 0x056a41 thru 0x060533 (0x9af3 bytes) 000014[0] 1746136064.441922 read 0x436c7c thru 0x4398eb (0x2c70 bytes) 000016[0] 1746136064.475458 read 0x453b99 thru 0x458ea5 (0x530d bytes) 000017[0] 1746136064.515178 mapread 0x4cac1d thru 0x4d1e0c (0x71f0 bytes) 000023[0] 1746136064.608030 mapwrite 0x6de1b6 thru 0x6ebf3f (0xdd8a bytes) 000026[1] 1746136064.789758 write 0x63b9c3 thru 0x6402f4 (0x4932 bytes) 000027[0] 1746136064.814561 mapwrite 0x794f2d thru 0x7a1e7e (0xcf52 bytes) 000028[1] 1746136064.959063 read 0x391694 thru 0x3996b6 (0x8023 bytes) 000029[1] 1746136064.996772 mapwrite 0x66cf6b thru 0x66fd2d (0x2dc3 bytes) 000031[1] 1746136065.102034 mapwrite 0x46c66d thru 0x4706e7 (0x407b bytes) 000032[0] 1746136065.202688 write 0x5b7974 thru 0x5b9118 (0x17a5 bytes) 000033[0] 1746136065.235597 trunc from 0x7a1e7f to 0x16222e (0x63fc50 bytes) truncating to largest ever: 0x82eb04 000038[0] 1746136065.346679 trunc from 0x16222f to 0x82eb03 (0x6cc8d5 bytes) 000039[0] 1746136065.368590 read 0x309d95 thru 0x30b7b4 (0x1a20 bytes) 000040[1] 1746136065.397541 mapread 0x5eca13 thru 0x5ef0c4 (0x26b2 bytes) 000044[1] 1746136065.434822 trunc from 0x82eb04 to 0x39dff9 (0x490b0a bytes) 000047[1] 1746136065.520567 read 0x06d733 thru 0x0794fc (0xbdca bytes) 000051[1] 1746136065.627754 write 0x69644f thru 0x6a0294 (0x9e46 bytes) 000054[1] 1746136065.643759 read 0x2d674f thru 0x2d80dd (0x198f bytes) 000058[0] 1746136065.670753 read 0x3455f4 thru 0x345ce2 (0x6ef bytes) 000064[1] 1746136065.847731 read 0x193c6e thru 0x19732b (0x36be bytes) 000066[1] 1746136065.873245 read 0x45dc0d thru 0x468a92 (0xae86 bytes) 000068[0] 1746136065.898398 write 0x6e598c thru 0x6e9e30 (0x44a5 bytes) 000071[1] 1746136065.938782 mapread 0x4f7dff thru 0x4fddf3 (0x5ff5 bytes) 000076[0] 1746136065.965592 write 0x4216b5 thru 0x4252b6 (0x3c02 bytes) 000077[0] 1746136065.993638 mapwrite 0x0080ec thru 0x00b346 (0x325b bytes) 000078[0] 1746136066.122340 read 0x424de5 thru 0x428540 (0x375c bytes) 000080[1] 1746136066.163965 mapwrite 0x1f0bfc thru 0x1f5d25 (0x512a bytes) 000081[1] 1746136066.308093 read 0x154198 thru 0x163f9c (0xfe05 bytes) 000083[0] 1746136066.337495 read 0x2d947a thru 0x2dbf42 (0x2ac9 bytes) 000085[0] 1746136066.383783 trunc from 0x6e9e31 to 0x45fd21 (0x28a10f bytes) truncating to largest ever: 0x894d52 000087[1] 1746136066.447558 trunc from 0x45fd22 to 0x894d51 (0x435030 bytes) truncating to largest ever: 0x8e2d0a 000088[1] 1746136066.487790 trunc from 0x894d52 to 0x8e2d09 (0x4dfb8 bytes) 000093[1] 1746136066.633040 mapwrite 0x3cfd62 thru 0x3db17d (0xb41c bytes) 000095[1] 1746136066.837312 trunc from 0x8e2d0a to 0x2be883 (0x624486 bytes) 000097[1] 1746136066.890994 mapread 0x08d5bd thru 0x09ba61 (0xe4a5 bytes) 000101[0] 1746136066.967181 trunc from 0x2be884 to 0x53c3b0 (0x27db2d bytes) 000103[0] 1746136067.013691 write 0x78764e thru 0x792db0 (0xb763 bytes) 000104[0] 1746136067.015603 write 0x1e0ca0 thru 0x1e1f11 (0x1272 bytes) 000105[1] 1746136067.055525 trunc from 0x792db1 to 0x16638a (0x62ca26 bytes) 000110[0] 1746136067.197497 mapwrite 0x4a15dd thru 0x4b0f9f (0xf9c3 bytes) 000112[1] 1746136067.389399 mapread 0x39eb5d thru 0x3a5704 (0x6ba8 bytes) 000113[0] 1746136067.428657 read 0x263cca thru 0x26c978 (0x8caf bytes) 000114[0] 1746136067.435270 mapread 0x241bd9 thru 0x248975 (0x6d9d bytes) 000115[1] 1746136067.485385 mapread 0x438294 thru 0x446da4 (0xeb11 bytes) 000116[0] 1746136067.529149 read 0x3c487f thru 0x3c9e1e (0x55a0 bytes) 000121[0] 1746136067.542849 mapwrite 0x03b605 thru 0x042f3d (0x7939 bytes) 000124[1] 1746136067.681048 mapread 0x04d5e8 thru 0x0594ec (0xbf05 bytes) 000128[0] 1746136067.732244 read 0x02d682 thru 0x035c37 (0x85b6 bytes) 000131[1] 1746136067.740078 trunc from 0x4b0fa0 to 0x47c891 (0x3470e bytes) 000134[1] 1746136067.788387 mapread 0x3442ab thru 0x34e128 (0x9e7e bytes) 000136[0] 1746136067.832815 mapread 0x109b4b thru 0x10f432 (0x58e8 bytes) 000137[1] 1746136067.869219 write 0x29e654 thru 0x2abbd2 (0xd57f bytes) 000139[0] 1746136067.931581 read 0x0e6b11 thru 0x0eb23d (0x472d bytes) 000142[1] 1746136068.006901 trunc from 0x47c892 to 0x2bee41 (0x1bda50 bytes) 000143[0] 1746136068.045852 trunc from 0x2bee42 to 0x2aee84 (0xffbd bytes) 000145[0] 1746136068.100169 read 0x22705e thru 0x236100 (0xf0a3 bytes) 000146[1] 1746136068.148529 write 0x423c92 thru 0x4314c7 (0xd836 bytes) 000148[0] 1746136068.224214 read 0x3e809c thru 0x3ee18a (0x60ef bytes) 000149[1] 1746136068.295914 mapread 0x41e7ba thru 0x4223c0 (0x3c07 bytes) 000152[1] 1746136068.368146 mapwrite 0x7cd5f4 thru 0x7cf3d2 (0x1ddf bytes) 000161[1] 1746136068.534206 write 0x4a2d4f thru 0x4b0393 (0xd645 bytes) 000164[0] 1746136068.574187 trunc from 0x7cf3d3 to 0x091b92 (0x73d840 bytes) 000167[1] 1746136068.652226 mapwrite 0x6ea2b4 thru 0x6ec0fa (0x1e47 bytes) 000168[1] 1746136068.785343 mapwrite 0x6e017e thru 0x6eee92 (0xed15 bytes) 000170[0] 1746136068.925619 read 0x5e0b7b thru 0x5f0372 (0xf7f8 bytes) 000179[1] 1746136068.989038 mapread 0x1ffe23 thru 0x20b536 (0xb714 bytes) 000180[0] 1746136069.005777 read 0x28ff2f thru 0x29c385 (0xc457 bytes) 000181[1] 1746136069.019975 mapread 0x1584a1 thru 0x15e797 (0x62f7 bytes) 000182[0] 1746136069.032976 mapwrite 0x178a84 thru 0x178c61 (0x1de bytes) 000183[0] 1746136069.085539 write 0x51ea65 thru 0x52a3da (0xb976 bytes) 000184[0] 1746136069.097238 write 0x6597aa thru 0x66218b (0x89e2 bytes) 000185[1] 1746136069.102395 trunc from 0x6eee93 to 0x762679 (0x737e7 bytes) 000187[0] 1746136069.155231 read 0x1e9166 thru 0x1f5497 (0xc332 bytes) 000189[0] 1746136069.167000 write 0x714f97 thru 0x71f111 (0xa17b bytes) 000192[1] 1746136069.189164 mapwrite 0x129748 thru 0x12cd72 (0x362b bytes) 000194[0] 1746136069.273184 write 0x4bc7a7 thru 0x4c9f63 (0xd7bd bytes) 000195[0] 1746136069.300635 trunc from 0x76267a to 0x24d16a (0x51550f bytes) 000196[0] 1746136069.337483 trunc from 0x24d16b to 0x73b579 (0x4ee40f bytes) 000197[0] 1746136069.374834 mapread 0x232d10 thru 0x239a53 (0x6d44 bytes) 000199[0] 1746136069.392836 write 0x3c6931 thru 0x3cb96d (0x503d bytes) truncating to largest ever: 0x97605d 000201[1] 1746136069.413375 trunc from 0x73b57a to 0x97605c (0x23aae3 bytes) 000207[0] 1746136069.500734 trunc from 0x97605d to 0x13b892 (0x83a7ca bytes) 000208[0] 1746136069.562159 trunc from 0x13b893 to 0x8751e9 (0x739957 bytes) 000209[0] 1746136069.599748 mapread 0x46a9d1 thru 0x478e92 (0xe4c2 bytes) 000210[1] 1746136069.637783 mapwrite 0x7e06c6 thru 0x7e48f1 (0x422c bytes) 000211[1] 1746136069.790297 mapread 0x2bd938 thru 0x2c92f5 (0xb9be bytes) 000214[1] 1746136069.885874 mapread 0x42cf14 thru 0x437139 (0xa226 bytes) 000215[0] 1746136069.910094 mapwrite 0x31801e thru 0x326e6f (0xee52 bytes) 000219[0] 1746136070.145848 mapread 0x219ff3 thru 0x21b6c2 (0x16d0 bytes) 000220[0] 1746136070.205363 mapread 0x3bf98a thru 0x3c3900 (0x3f77 bytes) 000221[1] 1746136070.271133 read 0x6ce85b thru 0x6dc2e3 (0xda89 bytes) 000222[1] 1746136070.291387 mapwrite 0x7af051 thru 0x7ba553 (0xb503 bytes) 000223[0] 1746136070.379257 mapread 0x64f7a6 thru 0x653311 (0x3b6c bytes) 000225[0] 1746136070.418106 read 0x52c447 thru 0x53b9a5 (0xf55f bytes) 000226[0] 1746136070.426420 mapwrite 0x79db17 thru 0x7a009d (0x2587 bytes) 000227[0] 1746136070.527388 trunc from 0x8751ea to 0x816f58 (0x5e291 bytes) 000228[1] 1746136070.575156 read 0x67e5d4 thru 0x681364 (0x2d91 bytes) 000229[0] 1746136070.612045 mapwrite 0x4149b8 thru 0x41765d (0x2ca6 bytes) 000231[1] 1746136070.738729 write 0x4f1171 thru 0x4fed2b (0xdbbb bytes) 000232[0] 1746136070.787063 trunc from 0x816f59 to 0x8f70cb (0xe0173 bytes) 000233[0] 1746136070.841017 read 0x2bf517 thru 0x2cc05d (0xcb47 bytes) 000234[1] 1746136070.886590 trunc from 0x8f70cc to 0x2a4bc9 (0x652502 bytes) 000237[0] 1746136070.945835 mapwrite 0x6d6b7e thru 0x6dec2f (0x80b2 bytes) 000238[0] 1746136071.085783 write 0x242bd7 thru 0x243f7c (0x13a6 bytes) 000239[0] 1746136071.100858 trunc from 0x6dec30 to 0x7d88fc (0xf9ccd bytes) 000242[1] 1746136071.138024 write 0x67c7ea thru 0x689232 (0xca49 bytes) 000244[1] 1746136071.183236 trunc from 0x7d88fd to 0x4c7fe3 (0x310919 bytes) 000246[1] 1746136071.276493 mapwrite 0x42af81 thru 0x4347dc (0x985c bytes) 000249[1] 1746136071.389946 trunc from 0x4c7fe4 to 0x7a5eb9 (0x2dded6 bytes) 000250[0] 1746136071.427927 trunc from 0x7a5eba to 0x049fbd (0x75befc bytes) 000253[1] 1746136071.459692 read 0x03bfda thru 0x049971 (0xd998 bytes) 000257[1] 1746136071.487162 trunc from 0x049fbe to 0x205c9a (0x1bbcdd bytes) 000259[1] 1746136071.547901 mapwrite 0x021eac thru 0x0309a5 (0xeafa bytes) 000261[1] 1746136071.644696 trunc from 0x205c9b to 0x5f4bbc (0x3eef22 bytes) 000262[0] 1746136071.661255 read 0x364958 thru 0x371aec (0xd195 bytes) 000264[1] 1746136071.801839 mapread 0x579318 thru 0x57bccb (0x29b4 bytes) 000266[1] 1746136071.841140 write 0x053aa0 thru 0x0638e8 (0xfe49 bytes) 000275[0] 1746136071.902167 mapread 0x5f05ef thru 0x5f4bbc (0x45ce bytes) 000277[0] 1746136071.969166 read 0x21418a thru 0x2217a9 (0xd620 bytes) 000278[1] 1746136071.994737 mapwrite 0x89d662 thru 0x8a5c0f (0x85ae bytes) 000279[0] 1746136072.151398 mapwrite 0x8a22c6 thru 0x8a7f56 (0x5c91 bytes) 000280[1] 1746136072.305581 trunc from 0x8a7f57 to 0x288755 (0x61f801 bytes) 000281[0] 1746136072.370257 write 0x419a69 thru 0x419ec0 (0x458 bytes) 000283[1] 1746136072.417844 mapwrite 0x651fb5 thru 0x65afc3 (0x900f bytes) 000290[1] 1746136072.651255 mapwrite 0x051c2c thru 0x05bf19 (0xa2ee bytes) 000293[1] 1746136072.845712 mapread 0x2a6b6c thru 0x2aa825 (0x3cba bytes) 000295[1] 1746136072.874839 mapwrite 0x94db14 thru 0x958bbd (0xb0aa bytes) 000297[1] 1746136073.034398 trunc from 0x958bbe to 0x18586d (0x7d3350 bytes) 000298[0] 1746136073.062701 write 0x5ef9ae thru 0x5f45e5 (0x4c38 bytes) 000300[0] 1746136073.103998 mapwrite 0x5e9c3d thru 0x5eb913 (0x1cd7 bytes) 000303[0] 1746136073.233904 write 0x0b4378 thru 0x0c0cda (0xc963 bytes) 000305[1] 1746136073.285719 mapread 0x53d5a8 thru 0x53e416 (0xe6f bytes) 000312[0] 1746136073.366036 read 0x02924e thru 0x029bd7 (0x98a bytes) 000313[1] 1746136073.373460 trunc from 0x5f45e6 to 0x4378c0 (0x1bcd25 bytes) 000314[0] 1746136073.436165 read 0x06c5b1 thru 0x06d699 (0x10e9 bytes) 000317[1] 1746136073.532357 trunc from 0x4378c1 to 0x7f2faa (0x3bb6ea bytes) 000318[0] 1746136073.578117 mapread 0x0a7b28 thru 0x0b57f1 (0xdcca bytes) 000319[1] 1746136073.655492 mapwrite 0x1359c4 thru 0x13e9a6 (0x8fe3 bytes) 000320[0] 1746136073.791128 write 0x536d8a thru 0x538921 (0x1b98 bytes) 000322[1] 1746136073.890189 trunc from 0x7f2fab to 0x18a101 (0x668ea9 bytes) 000323[0] 1746136073.961291 mapread 0x0f0898 thru 0x0f97b7 (0x8f20 bytes) 000324[1] 1746136074.026331 read 0x0e3364 thru 0x0ef363 (0xc000 bytes) 000325[1] 1746136074.075253 trunc from 0x18a102 to 0x06847c (0x121c85 bytes) 000326[1] 1746136074.118770 mapread 0x0104f1 thru 0x014c32 (0x4742 bytes) 000328[0] 1746136074.228505 trunc from 0x06847d to 0x3e423c (0x37bdc0 bytes) 000330[0] 1746136074.316362 write 0x4ff083 thru 0x50d1be (0xe13c bytes) 000331[0] 1746136074.318850 mapwrite 0x786b2e thru 0x78d147 (0x661a bytes) 000333[0] 1746136074.461210 mapread 0x1c83bf thru 0x1d50bf (0xcd01 bytes) 000336[0] 1746136074.553299 mapwrite 0x6d4f9f thru 0x6d9b91 (0x4bf3 bytes) 000337[0] 1746136074.640674 trunc from 0x78d148 to 0x134b12 (0x658635 bytes) 000338[1] 1746136074.713378 mapwrite 0x97550e thru 0x97ca0f (0x7502 bytes) 000339[1] 1746136074.993876 trunc from 0x97ca10 to 0x2703a1 (0x70c66e bytes) 000343[1] 1746136075.104515 trunc from 0x2703a2 to 0x69f848 (0x42f4a7 bytes) 000345[0] 1746136075.158211 read 0x152ff1 thru 0x158810 (0x5820 bytes) 000346[0] 1746136075.184723 mapwrite 0x8a0671 thru 0x8af222 (0xebb2 bytes) 000347[0] 1746136075.336356 trunc from 0x8af223 to 0x0baa97 (0x7f478b bytes) 000349[1] 1746136075.347029 write 0x634a91 thru 0x643332 (0xe8a2 bytes) 000350[1] 1746136075.374396 read 0x0ff7c9 thru 0x10f5ac (0xfde4 bytes) 000352[0] 1746136075.539043 mapwrite 0x91d1f3 thru 0x92880f (0xb61d bytes) 000357[1] 1746136075.802116 trunc from 0x928810 to 0x554b2f (0x3d3ce0 bytes) 000364[1] 1746136075.905061 read 0x250bf0 thru 0x258750 (0x7b61 bytes) 000367[0] 1746136075.923724 trunc from 0x554b30 to 0x10f1df (0x445950 bytes) 000369[1] 1746136076.026918 mapread 0x038231 thru 0x040564 (0x8334 bytes) 000370[1] 1746136076.086046 read 0x05c66c thru 0x0660ad (0x9a42 bytes) 000371[0] 1746136076.131647 mapread 0x07b094 thru 0x085a12 (0xa97f bytes) 000375[0] 1746136076.197511 mapread 0x09d508 thru 0x0a6d2c (0x9825 bytes) 000376[0] 1746136076.226888 mapwrite 0x35bac0 thru 0x36a008 (0xe549 bytes) 000377[0] 1746136076.424675 mapread 0x16724b thru 0x16d42c (0x61e2 bytes) 000378[1] 1746136076.448689 mapread 0x0f278d thru 0x0fba7e (0x92f2 bytes) 000380[1] 1746136076.577338 mapread 0x034a1f thru 0x03bf17 (0x74f9 bytes) 000381[1] 1746136076.635452 read 0x09af82 thru 0x0a11f8 (0x6277 bytes) 000383[1] 1746136076.674519 read 0x321bcb thru 0x32ac4b (0x9081 bytes) 000385[0] 1746136076.695596 mapwrite 0x07dec3 thru 0x0820d7 (0x4215 bytes) 000386[0] 1746136076.803265 read 0x1916e7 thru 0x1936cc (0x1fe6 bytes) 000388[1] 1746136076.879639 write 0x6addcd thru 0x6b503d (0x7271 bytes) 000389[0] 1746136076.894758 write 0x0fc9b7 thru 0x0ff00c (0x2656 bytes) 000392[0] 1746136076.924889 trunc from 0x6b503e to 0x4a8697 (0x20c9a6 bytes) 000393[1] 1746136076.961701 mapread 0x25a775 thru 0x25b3cf (0xc5b bytes) 000398[1] 1746136077.144551 write 0x0511e4 thru 0x051bf3 (0xa10 bytes) 000399[0] 1746136077.208315 mapread 0x161280 thru 0x16c447 (0xb1c8 bytes) 000400[0] 1746136077.357348 write 0x48ae48 thru 0x499339 (0xe4f2 bytes) 000401[1] 1746136077.396730 mapread 0x2fd34c thru 0x30bfbf (0xec74 bytes) 000404[0] 1746136077.468599 read 0x44f789 thru 0x457b2f (0x83a7 bytes) 000406[1] 1746136077.485242 mapread 0x3607b4 thru 0x37007c (0xf8c9 bytes) 000407[1] 1746136077.581639 trunc from 0x4a8698 to 0x91f755 (0x4770be bytes) 000409[1] 1746136077.646804 mapwrite 0x3484f3 thru 0x34e8c0 (0x63ce bytes) 000412[1] 1746136077.797556 write 0x42851d thru 0x42b1fc (0x2ce0 bytes) 000414[0] 1746136077.904440 read 0x608071 thru 0x616095 (0xe025 bytes) 000416[0] 1746136077.937955 mapwrite 0x7350eb thru 0x73df2b (0x8e41 bytes) 000418[1] 1746136078.071874 write 0x8154c1 thru 0x824ba7 (0xf6e7 bytes) 000419[0] 1746136078.116638 trunc from 0x91f756 to 0x3dbf28 (0x54382d bytes) 000420[1] 1746136078.179830 read 0x1f0249 thru 0x1fd110 (0xcec8 bytes) 000425[1] 1746136078.294628 trunc from 0x3dbf29 to 0x14b4aa (0x290a7e bytes) 000426[0] 1746136078.370554 mapwrite 0x0255e7 thru 0x0311a2 (0xbbbc bytes) 000431[1] 1746136078.544117 mapread 0x030cc8 thru 0x03e0c7 (0xd400 bytes) 000432[1] 1746136078.639076 read 0x12361a thru 0x126ce3 (0x36ca bytes) 000433[1] 1746136078.676211 mapread 0x0ac2de thru 0x0b8e29 (0xcb4c bytes) 000435[1] 1746136078.705302 mapread 0x044fac thru 0x04979b (0x47f0 bytes) 000437[1] 1746136078.732753 read 0x0b911d thru 0x0c6a15 (0xd8f9 bytes) 000438[0] 1746136078.749128 mapread 0x067f7c thru 0x06e6c8 (0x674d bytes) 000439[1] 1746136078.774990 read 0x01173d thru 0x0196d4 (0x7f98 bytes) 000441[0] 1746136078.803960 mapwrite 0x08e050 thru 0x09cc6f (0xec20 bytes) 000442[1] 1746136078.956172 read 0x046d96 thru 0x04e885 (0x7af0 bytes) 000443[0] 1746136079.004525 mapread 0x0a6e1d thru 0x0aec7f (0x7e63 bytes) 000445[0] 1746136079.071193 trunc from 0x14b4ab to 0x219da9 (0xce8ff bytes) 000448[0] 1746136079.171601 mapread 0x06bfba thru 0x072c15 (0x6c5c bytes) 000452[1] 1746136079.203506 trunc from 0x219daa to 0x28b34c (0x715a3 bytes) 000453[1] 1746136079.252557 mapread 0x1cde76 thru 0x1dac50 (0xcddb bytes) 000454[0] 1746136079.277757 write 0x357629 thru 0x35dcce (0x66a6 bytes) 000456[1] 1746136079.335055 mapread 0x0b22d1 thru 0x0b9999 (0x76c9 bytes) 000457[0] 1746136079.436710 mapread 0x3528f4 thru 0x353ea6 (0x15b3 bytes) 000458[1] 1746136079.454316 mapread 0x1d0009 thru 0x1d4f1e (0x4f16 bytes) 000463[1] 1746136079.532060 read 0x222543 thru 0x22b411 (0x8ecf bytes) 000466[1] 1746136079.612555 trunc from 0x35dccf to 0x853152 (0x4f5484 bytes) 000468[0] 1746136079.693314 mapwrite 0x321041 thru 0x323a15 (0x29d5 bytes) 000469[1] 1746136079.818306 read 0x2b1d62 thru 0x2b9a0b (0x7caa bytes) 000470[0] 1746136079.839716 write 0x7bb4e4 thru 0x7c9e6b (0xe988 bytes) 000471[0] 1746136079.876700 mapwrite 0x653d88 thru 0x65d47a (0x96f3 bytes) 000472[1] 1746136080.010955 mapread 0x675207 thru 0x6820ed (0xcee7 bytes) 000473[1] 1746136080.069062 write 0x64b30c thru 0x657bcb (0xc8c0 bytes) 000477[0] 1746136080.227372 trunc from 0x853153 to 0x330d10 (0x522442 bytes) 000478[0] 1746136080.366415 trunc from 0x330d11 to 0x01e0d6 (0x312c3a bytes) 000480[0] 1746136080.429514 read 0x016ffc thru 0x01e0d6 (0x70db bytes) 000482[1] 1746136080.492022 write 0x053cf2 thru 0x06164b (0xd95a bytes) 000483[0] 1746136080.511713 write 0x84456c thru 0x84cbfb (0x8690 bytes) 000486[1] 1746136080.567266 read 0x788433 thru 0x79457b (0xc149 bytes) 000487[1] 1746136080.666793 read 0x67fe95 thru 0x684f00 (0x506c bytes) 000493[1] 1746136080.794733 mapwrite 0x215dbd thru 0x21a9fc (0x4c40 bytes) 000495[1] 1746136080.914879 mapwrite 0x7939c6 thru 0x7a073f (0xcd7a bytes) 000497[0] 1746136081.127535 mapread 0x5bcc4d thru 0x5c4a32 (0x7de6 bytes) 000499[1] 1746136081.183670 trunc from 0x84cbfc to 0x1873a2 (0x6c5859 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 6081 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x6bd653 000009[0] 1746136082.136123 trunc from 00000000 to 0x6bd652 (0x6bd653 bytes) 000012[1] 1746136082.168437 mapwrite 0x1fb000 thru 0x1fbbff (0xc00 bytes) 000014[0] 1746136082.326180 write_OD 0x62f000 thru 0x632fff (0x4000 bytes) 000016[0] 1746136082.426631 write_OD 0x60b000 thru 0x60dfff (0x3000 bytes) 000018[1] 1746136082.523358 mapwrite 0x54d000 thru 0x55c5fe (0xf5ff bytes) 000020[1] 1746136082.703156 mapwrite 0x260000 thru 0x26d28a (0xd28b bytes) 000025[1] 1746136082.857388 read 0x498000 thru 0x4a31e9 (0xb1ea bytes) 000028[1] 1746136082.880713 write 0x1f1000 thru 0x1fa08c (0x908d bytes) 000031[0] 1746136082.911655 write_OD 0x3a6000 thru 0x3abfff (0x6000 bytes) 000034[1] 1746136083.027781 mapread 0x58e000 thru 0x59a7d4 (0xc7d5 bytes) 000036[0] 1746136083.048140 mapwrite 0x1a1000 thru 0x1acd49 (0xbd4a bytes) 000038[1] 1746136083.168163 mapwrite 0x518000 thru 0x51928f (0x1290 bytes) truncating to largest ever: 0x9a2654 000039[1] 1746136083.262244 trunc from 0x6bd653 to 0x9a2653 (0x2e5001 bytes) 000042[0] 1746136083.305155 trunc from 0x9a2654 to 0x4c8649 (0x4da00a bytes) 000044[0] 1746136083.356158 read_OD 0x0b7000 thru 0x0bdfff (0x7000 bytes) 000046[0] 1746136083.383425 mapread 0x2b0000 thru 0x2b3553 (0x3554 bytes) 000047[0] 1746136083.399255 trunc from 0x4c864a to 0x506081 (0x3da38 bytes) 000049[1] 1746136083.441386 mapread 0x1a2000 thru 0x1a6f78 (0x4f79 bytes) 000050[0] 1746136083.508306 mapread 0x08f000 thru 0x09ef34 (0xff35 bytes) 000051[1] 1746136083.540283 read 0x333000 thru 0x3373fe (0x43ff bytes) 000052[1] 1746136083.549574 mapwrite 0x857000 thru 0x860757 (0x9758 bytes) 000053[0] 1746136083.648386 mapread 0x37a000 thru 0x380745 (0x6746 bytes) 000062[0] 1746136083.685532 mapread 0x0f6000 thru 0x0f9760 (0x3761 bytes) 000065[1] 1746136083.701218 write 0x73a000 thru 0x73d771 (0x3772 bytes) 000066[1] 1746136083.715165 mapwrite 0x1e1000 thru 0x1ed591 (0xc592 bytes) 000067[0] 1746136083.807668 read_OD 0x334000 thru 0x33bfff (0x8000 bytes) 000068[1] 1746136083.832094 mapread 0x85d000 thru 0x85fd34 (0x2d35 bytes) 000069[0] 1746136083.849356 mapread 0x1a0000 thru 0x1a96fb (0x96fc bytes) 000071[0] 1746136083.860958 mapwrite 0x7a6000 thru 0x7ae44c (0x844d bytes) 000072[1] 1746136083.960237 mapread 0x43c000 thru 0x442cf5 (0x6cf6 bytes) 000076[0] 1746136083.998023 trunc from 0x860758 to 0x1a62d7 (0x6ba480 bytes) 000079[1] 1746136084.069912 mapwrite 0x5e8000 thru 0x5f0e29 (0x8e2a bytes) 000083[1] 1746136084.191892 read 0x40e000 thru 0x415664 (0x7665 bytes) 000084[1] 1746136084.198976 write 0x5c8000 thru 0x5cbbf8 (0x3bf9 bytes) 000085[1] 1746136084.204553 mapread 0x230000 thru 0x2305af (0x5b0 bytes) 000089[0] 1746136084.242273 trunc from 0x5f0e2a to 0x1a57a1 (0x44b688 bytes) 000091[1] 1746136084.291094 mapread 0x0e5000 thru 0x0f262b (0xd62c bytes) 000092[1] 1746136084.330784 write 0x84f000 thru 0x85c2f3 (0xd2f4 bytes) 000094[0] 1746136084.356105 mapread 0x71a000 thru 0x72769a (0xd69b bytes) 000096[0] 1746136084.395362 trunc from 0x85c2f4 to 0x2348ab (0x627a48 bytes) 000097[1] 1746136084.419575 mapwrite 0x5ab000 thru 0x5b12ec (0x62ed bytes) 000100[0] 1746136084.575257 mapwrite 0x939000 thru 0x945931 (0xc932 bytes) 000101[0] 1746136084.705744 mapread 0x2e4000 thru 0x2f3cce (0xfccf bytes) 000106[1] 1746136084.772401 mapwrite 0x266000 thru 0x26ce62 (0x6e63 bytes) 000107[1] 1746136084.874076 write 0x4dc000 thru 0x4e4559 (0x855a bytes) 000114[0] 1746136084.920341 read_OD 0x086000 thru 0x091fff (0xc000 bytes) 000115[1] 1746136084.952077 mapread 0x421000 thru 0x42fd54 (0xed55 bytes) 000116[1] 1746136084.970247 mapwrite 0x85e000 thru 0x85f2e8 (0x12e9 bytes) 000117[1] 1746136085.074831 write 0x982000 thru 0x983975 (0x1976 bytes) 000118[1] 1746136085.075476 mapread 0x5a9000 thru 0x5aa360 (0x1361 bytes) 000119[1] 1746136085.105172 write 0x683000 thru 0x686d48 (0x3d49 bytes) 000123[1] 1746136085.115540 mapread 0x884000 thru 0x88c586 (0x8587 bytes) 000126[1] 1746136085.166848 mapread 0x11d000 thru 0x12832b (0xb32c bytes) 000130[0] 1746136085.208994 read_OD 0x11c000 thru 0x11efff (0x3000 bytes) 000131[0] 1746136085.234948 trunc from 0x983976 to 0x0cd17b (0x8b67fa bytes) 000134[1] 1746136085.273552 mapwrite 0x7ff000 thru 0x80d347 (0xe348 bytes) 000135[0] 1746136085.422220 mapread 0x3b1000 thru 0x3bcaec (0xbaed bytes) 000136[1] 1746136085.474687 mapread 0x0e8000 thru 0x0ead86 (0x2d87 bytes) 000137[1] 1746136085.502064 trunc from 0x80d348 to 0x7d5ed3 (0x37474 bytes) 000139[1] 1746136085.553305 read 0x243000 thru 0x24657c (0x357d bytes) 000140[0] 1746136085.572499 write_OD 0x383000 thru 0x383fff (0x1000 bytes) 000144[1] 1746136085.657301 read 0x7c0000 thru 0x7c93b5 (0x93b6 bytes) 000145[0] 1746136085.664629 mapread 0x281000 thru 0x284388 (0x3389 bytes) 000146[1] 1746136085.682788 read 0x11a000 thru 0x121c80 (0x7c81 bytes) 000147[1] 1746136085.694551 trunc from 0x7d5ed4 to 0x381a52 (0x454481 bytes) 000149[1] 1746136085.748778 write 0x100000 thru 0x102239 (0x223a bytes) 000151[1] 1746136085.796808 mapread 0x2ba000 thru 0x2c8a24 (0xea25 bytes) 000152[0] 1746136085.844481 mapread 0x1c4000 thru 0x1d2e15 (0xee16 bytes) 000157[1] 1746136085.901121 trunc from 0x381a53 to 0x4d4529 (0x152ad7 bytes) 000158[1] 1746136085.936366 mapread 0x10f000 thru 0x112194 (0x3195 bytes) 000162[1] 1746136085.971377 read 0x2b9000 thru 0x2c59d8 (0xc9d9 bytes) 000163[0] 1746136085.981728 trunc from 0x4d452a to 0x942744 (0x46e21b bytes) 000164[0] 1746136086.058088 mapwrite 0x8cf000 thru 0x8d9c78 (0xac79 bytes) 000168[1] 1746136086.304040 read 0x137000 thru 0x146c4e (0xfc4f bytes) 000169[1] 1746136086.346165 read 0x486000 thru 0x48a19c (0x419d bytes) 000171[0] 1746136086.352016 write_OD 0x7cd000 thru 0x7d3fff (0x7000 bytes) 000173[1] 1746136086.425484 mapread 0x70d000 thru 0x71cf3f (0xff40 bytes) 000175[1] 1746136086.461558 write 0x1bb000 thru 0x1bfc06 (0x4c07 bytes) 000176[0] 1746136086.507199 trunc from 0x942745 to 0x0f3556 (0x84f1ee bytes) 000178[0] 1746136086.545802 mapread 0x00b000 thru 0x01035b (0x535c bytes) 000179[0] 1746136086.584646 read_OD 0x096000 thru 0x096fff (0x1000 bytes) 000180[1] 1746136086.597783 read 0x0a7000 thru 0x0b267b (0xb67c bytes) 000181[0] 1746136086.627174 trunc from 0x0f3557 to 0x86946c (0x775f16 bytes) 000183[0] 1746136086.705302 write_OD 0x7bb000 thru 0x7c0fff (0x6000 bytes) 000184[0] 1746136086.801905 write_OD 0x4cc000 thru 0x4cffff (0x4000 bytes) 000185[0] 1746136086.874402 write_OD 0x8d4000 thru 0x8d8fff (0x5000 bytes) 000186[1] 1746136086.950240 trunc from 0x8d9000 to 0x51508d (0x3c3f72 bytes) 000188[0] 1746136086.977775 trunc from 0x51508e to 0x1d3e71 (0x34121c bytes) 000191[1] 1746136087.026395 trunc from 0x1d3e72 to 0x984a13 (0x7b0ba2 bytes) 000193[1] 1746136087.061310 mapwrite 0x558000 thru 0x55ba5b (0x3a5c bytes) 000195[0] 1746136087.281756 mapwrite 0x7f7000 thru 0x8024db (0xb4dc bytes) 000197[1] 1746136087.564925 mapread 0x09f000 thru 0x0a4211 (0x5212 bytes) 000198[1] 1746136087.659327 write 0x3d2000 thru 0x3d80b5 (0x60b6 bytes) 000199[0] 1746136087.677350 mapread 0x033000 thru 0x042dde (0xfddf bytes) 000200[1] 1746136087.781624 read 0x5b9000 thru 0x5bb39e (0x239f bytes) 000202[1] 1746136087.797222 mapread 0x014000 thru 0x01a0fa (0x60fb bytes) 000204[1] 1746136087.822337 trunc from 0x984a14 to 0x68b49d (0x2f9576 bytes) 000205[0] 1746136087.850924 mapread 0x594000 thru 0x5a24b2 (0xe4b3 bytes) 000207[1] 1746136087.936167 trunc from 0x68b49e to 0x1dff1e (0x4ab57f bytes) 000211[1] 1746136087.953308 read 0x1a9000 thru 0x1b5363 (0xc364 bytes) 000212[1] 1746136088.004612 mapwrite 0x00a000 thru 0x014ad3 (0xaad4 bytes) 000213[0] 1746136088.112860 trunc from 0x1dff1f to 0x33fe40 (0x15ff22 bytes) 000215[1] 1746136088.182915 write 0x0cb000 thru 0x0d4ecf (0x9ed0 bytes) 000217[0] 1746136088.251661 trunc from 0x33fe41 to 0x8e9a6b (0x5a9c2b bytes) truncating to largest ever: 0x9a2d65 000222[0] 1746136088.346925 trunc from 0x8e9a6c to 0x9a2d64 (0xb92f9 bytes) 000224[0] 1746136088.368763 read_OD 0x83b000 thru 0x83cfff (0x2000 bytes) 000226[0] 1746136088.396791 read_OD 0x82e000 thru 0x835fff (0x8000 bytes) 000227[0] 1746136088.409685 mapwrite 0x52a000 thru 0x53106e (0x706f bytes) 000228[0] 1746136088.516924 mapwrite 0x61a000 thru 0x622b9d (0x8b9e bytes) 000231[1] 1746136088.631160 mapread 0x757000 thru 0x75e207 (0x7208 bytes) 000232[1] 1746136088.760885 write 0x0c5000 thru 0x0cc35d (0x735e bytes) 000234[1] 1746136088.811641 read 0x749000 thru 0x7583f9 (0xf3fa bytes) 000236[1] 1746136088.832815 mapwrite 0x99f000 thru 0x9ad317 (0xe318 bytes) 000237[1] 1746136088.961847 mapwrite 0x1ef000 thru 0x1fb9a3 (0xc9a4 bytes) 000239[0] 1746136089.056164 trunc from 0x9ad318 to 0x036ac4 (0x976853 bytes) 000241[1] 1746136089.099636 write 0x35c000 thru 0x36a7b9 (0xe7ba bytes) 000243[1] 1746136089.109736 trunc from 0x36a7ba to 0x6fd2b9 (0x392b00 bytes) 000244[0] 1746136089.142673 trunc from 0x6fd2ba to 0x1ddf46 (0x51f373 bytes) 000245[0] 1746136089.181370 read_OD 0x153000 thru 0x160fff (0xe000 bytes) 000248[1] 1746136089.216229 write 0x2f8000 thru 0x2fffe4 (0x7fe5 bytes) 000249[1] 1746136089.238300 trunc from 0x2fffe5 to 0x591481 (0x29149d bytes) 000250[0] 1746136089.273458 mapread 0x0e3000 thru 0x0e7a2f (0x4a30 bytes) 000253[0] 1746136089.392388 mapread 0x1c7000 thru 0x1c9668 (0x2669 bytes) 000254[1] 1746136089.397369 read 0x174000 thru 0x175e1a (0x1e1b bytes) 000255[1] 1746136089.406401 mapread 0x48d000 thru 0x49136e (0x436f bytes) 000258[0] 1746136089.447211 mapwrite 0x2ac000 thru 0x2b78e9 (0xb8ea bytes) 000259[0] 1746136089.632621 trunc from 0x591482 to 0x447e85 (0x1495fc bytes) 000261[1] 1746136089.693805 trunc from 0x447e86 to 0x4a3acd (0x5bc48 bytes) 000262[1] 1746136089.733229 trunc from 0x4a3ace to 0x22620f (0x27d8be bytes) 000264[1] 1746136089.779425 trunc from 0x226210 to 0x11fb05 (0x10670a bytes) 000265[1] 1746136089.794316 read 0x066000 thru 0x06db5e (0x7b5f bytes) 000266[1] 1746136089.824463 write 0x7e5000 thru 0x7ee69b (0x969c bytes) 000267[1] 1746136089.853091 mapread 0x159000 thru 0x162520 (0x9521 bytes) 000270[1] 1746136089.940596 mapwrite 0x0de000 thru 0x0e977b (0xb77c bytes) 000274[1] 1746136090.073315 mapwrite 0x27f000 thru 0x2898f8 (0xa8f9 bytes) 000276[0] 1746136090.183744 read_OD 0x0b0000 thru 0x0befff (0xf000 bytes) 000279[0] 1746136090.228666 trunc from 0x7ee69c to 0x98a533 (0x19be98 bytes) 000280[1] 1746136090.301913 trunc from 0x98a534 to 0x34ea60 (0x63bad3 bytes) 000283[0] 1746136090.386057 trunc from 0x34ea61 to 0x559135 (0x20a6d5 bytes) 000286[0] 1746136090.497388 read_OD 0x49d000 thru 0x4a2fff (0x6000 bytes) 000287[0] 1746136090.524085 write_OD 0x414000 thru 0x41afff (0x7000 bytes) 000288[0] 1746136090.606659 mapwrite 0x983000 thru 0x98da05 (0xaa06 bytes) 000289[0] 1746136090.735458 trunc from 0x98da06 to 0x338491 (0x655574 bytes) 000294[0] 1746136090.779069 read_OD 0x19e000 thru 0x19efff (0x1000 bytes) 000296[1] 1746136090.791670 read 0x2f2000 thru 0x2f7326 (0x5327 bytes) 000297[1] 1746136090.859840 mapwrite 0x55e000 thru 0x565e28 (0x7e29 bytes) 000301[0] 1746136091.058419 read_OD 0x344000 thru 0x344fff (0x1000 bytes) 000302[1] 1746136091.107222 read 0x346000 thru 0x34bd6c (0x5d6d bytes) 000305[1] 1746136091.127485 write 0x563000 thru 0x5631af (0x1b0 bytes) 000306[0] 1746136091.162471 write_OD 0x059000 thru 0x05bfff (0x3000 bytes) 000307[1] 1746136091.224276 mapread 0x38d000 thru 0x397d4a (0xad4b bytes) 000309[1] 1746136091.236124 write 0x486000 thru 0x48aa71 (0x4a72 bytes) 000312[1] 1746136091.255651 read 0x249000 thru 0x24e98e (0x598f bytes) 000314[0] 1746136091.273870 write_OD 0x0c5000 thru 0x0d0fff (0xc000 bytes) 000315[0] 1746136091.363862 write_OD 0x9bf000 thru 0x9c9fff (0xb000 bytes) 000317[0] 1746136091.426753 trunc from 0x9ca000 to 0x00cb8d (0x9bd472 bytes) 000319[1] 1746136091.463753 write 0x226000 thru 0x230568 (0xa569 bytes) 000322[1] 1746136091.521667 mapwrite 0x496000 thru 0x49f507 (0x9508 bytes) 000323[0] 1746136091.721876 write_OD 0x3f6000 thru 0x403fff (0xe000 bytes) 000325[0] 1746136091.822672 write_OD 0x270000 thru 0x27efff (0xf000 bytes) 000328[1] 1746136091.916399 mapread 0x31b000 thru 0x32ad10 (0xfd11 bytes) 000329[0] 1746136091.946879 write_OD 0x5ef000 thru 0x5effff (0x1000 bytes) 000330[0] 1746136092.034267 read_OD 0x228000 thru 0x230fff (0x9000 bytes) 000331[1] 1746136092.042454 mapread 0x4bc000 thru 0x4caa26 (0xea27 bytes) 000333[1] 1746136092.086653 read 0x2f7000 thru 0x2fd6bc (0x66bd bytes) 000334[0] 1746136092.094389 mapwrite 0x797000 thru 0x7a303b (0xc03c bytes) 000336[0] 1746136092.284265 mapread 0x44e000 thru 0x457510 (0x9511 bytes) 000337[1] 1746136092.321779 mapread 0x15d000 thru 0x15f3d9 (0x23da bytes) 000341[1] 1746136092.377332 mapwrite 0x44c000 thru 0x45b68d (0xf68e bytes) 000345[1] 1746136092.609965 read 0x312000 thru 0x320eb8 (0xeeb9 bytes) 000346[1] 1746136092.640822 write 0x215000 thru 0x215049 (0x4a bytes) 000347[0] 1746136092.645286 trunc from 0x7a303c to 0x54e238 (0x254e03 bytes) 000348[1] 1746136092.745995 mapread 0x04c000 thru 0x058cf8 (0xccf9 bytes) 000350[1] 1746136092.792094 trunc from 0x54e239 to 0x4df23c (0x6effc bytes) 000353[0] 1746136092.873637 trunc from 0x4df23d to 0x97e401 (0x49f1c5 bytes) 000358[1] 1746136092.904024 mapwrite 0x9ea000 thru 0x9f39d4 (0x99d5 bytes) 000359[1] 1746136093.078267 mapread 0x681000 thru 0x68f2c1 (0xe2c2 bytes) 000360[1] 1746136093.104537 trunc from 0x9f39d5 to 0x116418 (0x8dd5bc bytes) 000363[1] 1746136093.166762 mapread 0x0ca000 thru 0x0d2681 (0x8682 bytes) 000364[1] 1746136093.198788 mapread 0x00d000 thru 0x01c6d1 (0xf6d2 bytes) 000366[0] 1746136093.231783 trunc from 0x116419 to 0x28ecf7 (0x1788df bytes) 000367[0] 1746136093.265626 read_OD 0x16d000 thru 0x16efff (0x2000 bytes) 000368[0] 1746136093.289236 read_OD 0x1bb000 thru 0x1c2fff (0x8000 bytes) 000371[0] 1746136093.300868 trunc from 0x28ecf8 to 0x900e76 (0x67217f bytes) 000372[1] 1746136093.345865 mapwrite 0x1e7000 thru 0x1edcdc (0x6cdd bytes) 000375[1] 1746136093.563637 mapwrite 0x34a000 thru 0x359320 (0xf321 bytes) 000377[1] 1746136093.685862 read 0x644000 thru 0x64a213 (0x6214 bytes) 000382[0] 1746136093.697167 mapwrite 0x8da000 thru 0x8dfe74 (0x5e75 bytes) 000383[1] 1746136093.960313 write 0x935000 thru 0x93e0a7 (0x90a8 bytes) 000384[0] 1746136093.999232 write_OD 0x062000 thru 0x06ffff (0xe000 bytes) 000385[1] 1746136094.166878 mapread 0x235000 thru 0x243acf (0xead0 bytes) 000387[1] 1746136094.203051 trunc from 0x93e0a8 to 0x894d4d (0xa935a bytes) 000390[0] 1746136094.255061 mapwrite 0x8dd000 thru 0x8de3fc (0x13fd bytes) 000391[1] 1746136094.389216 mapwrite 0x7d9000 thru 0x7e5b20 (0xcb21 bytes) 000392[1] 1746136094.493199 trunc from 0x8de3fd to 0x3a24a7 (0x53bf55 bytes) 000394[1] 1746136094.539734 write 0x11b000 thru 0x11eb31 (0x3b32 bytes) 000395[1] 1746136094.559243 write 0x6aa000 thru 0x6ac01f (0x2020 bytes) 000396[0] 1746136094.566122 read_OD 0x571000 thru 0x57efff (0xe000 bytes) 000398[1] 1746136094.593458 mapwrite 0x63b000 thru 0x6420a5 (0x70a6 bytes) 000399[1] 1746136094.747815 write 0x9ee000 thru 0x9f54f9 (0x74fa bytes) 000400[0] 1746136094.749463 write_OD 0x34a000 thru 0x34bfff (0x2000 bytes) 000401[1] 1746136094.836045 trunc from 0x9f54fa to 0x3e5991 (0x60fb68 bytes) 000403[0] 1746136094.884759 mapread 0x19f000 thru 0x1a396b (0x496c bytes) 000406[0] 1746136095.004787 mapwrite 0x95b000 thru 0x966eca (0xbecb bytes) 000407[0] 1746136095.277755 mapread 0x07a000 thru 0x089a66 (0xfa67 bytes) 000408[0] 1746136095.425594 write_OD 0x42f000 thru 0x437fff (0x9000 bytes) 000409[0] 1746136095.541058 read_OD 0x114000 thru 0x116fff (0x3000 bytes) 000410[0] 1746136095.549167 mapread 0x070000 thru 0x07752b (0x752c bytes) 000412[0] 1746136095.562720 write_OD 0x563000 thru 0x56dfff (0xb000 bytes) 000414[1] 1746136095.656431 mapwrite 0x7b2000 thru 0x7bb612 (0x9613 bytes) 000415[1] 1746136095.803466 write 0x5a3000 thru 0x5aaea1 (0x7ea2 bytes) 000416[0] 1746136095.816212 trunc from 0x966ecb to 0x05f20d (0x907cbd bytes) 000419[0] 1746136095.885022 read_OD 0x04b000 thru 0x055fff (0xb000 bytes) 000420[0] 1746136095.904182 read_OD 0x022000 thru 0x024fff (0x3000 bytes) 000425[1] 1746136095.913383 mapwrite 0x027000 thru 0x032f8a (0xbf8b bytes) 000426[0] 1746136096.039242 mapwrite 0x4c2000 thru 0x4ce253 (0xc254 bytes) 000427[0] 1746136096.241694 write_OD 0x857000 thru 0x85efff (0x8000 bytes) 000428[0] 1746136096.307226 mapread 0x326000 thru 0x330f1f (0xaf20 bytes) 000431[0] 1746136096.355960 mapwrite 0x252000 thru 0x25f03f (0xd040 bytes) 000432[1] 1746136096.499536 mapwrite 0x35f000 thru 0x35f4d2 (0x4d3 bytes) 000434[1] 1746136096.577372 mapwrite 0x2f4000 thru 0x2ffbe8 (0xbbe9 bytes) 000438[0] 1746136096.704885 mapread 0x6a7000 thru 0x6b39dc (0xc9dd bytes) 000441[0] 1746136096.822711 trunc from 0x85f000 to 0x338d1b (0x5262e4 bytes) 000444[0] 1746136096.881106 trunc from 0x338d1c to 0x8eef9a (0x5b627f bytes) 000445[1] 1746136096.912495 write 0x4e9000 thru 0x4f03f1 (0x73f2 bytes) 000446[1] 1746136096.929812 trunc from 0x8eef9b to 0x1bae0a (0x734190 bytes) 000447[0] 1746136096.951104 mapwrite 0x300000 thru 0x30c1f7 (0xc1f8 bytes) 000449[1] 1746136097.144604 mapwrite 0x759000 thru 0x75fc1c (0x6c1d bytes) 000450[1] 1746136097.324644 mapread 0x6f3000 thru 0x6f80a0 (0x50a1 bytes) 000451[0] 1746136097.340434 mapwrite 0x863000 thru 0x86db52 (0xab53 bytes) 000455[1] 1746136097.534834 trunc from 0x86db53 to 0x1bd4bc (0x6b0696 bytes) 000456[1] 1746136097.605464 mapwrite 0x398000 thru 0x39ccac (0x4cad bytes) 000457[0] 1746136097.770625 mapwrite 0x894000 thru 0x8a1db0 (0xddb1 bytes) 000462[1] 1746136098.017411 trunc from 0x8a1db1 to 0x179ab5 (0x7282fb bytes) 000464[1] 1746136098.085223 trunc from 0x179ab6 to 0x71dd4a (0x5a4295 bytes) 000465[1] 1746136098.125475 trunc from 0x71dd4b to 0x6b34a3 (0x6a8a7 bytes) 000466[0] 1746136098.152091 read_OD 0x0f3000 thru 0x0fafff (0x8000 bytes) 000467[1] 1746136098.217854 read 0x107000 thru 0x10f15d (0x815e bytes) 000471[1] 1746136098.234620 read 0x3c2000 thru 0x3c6907 (0x4908 bytes) 000472[1] 1746136098.261295 read 0x39b000 thru 0x3a1eb7 (0x6eb8 bytes) 000473[0] 1746136098.274861 mapwrite 0x2a7000 thru 0x2a831e (0x131f bytes) 000479[0] 1746136098.468364 mapread 0x472000 thru 0x4783bb (0x63bc bytes) 000480[0] 1746136098.510851 read_OD 0x580000 thru 0x58bfff (0xc000 bytes) 000481[1] 1746136098.543715 read 0x489000 thru 0x4907e4 (0x77e5 bytes) 000482[0] 1746136098.550888 mapread 0x67f000 thru 0x68945d (0xa45e bytes) 000485[0] 1746136098.579333 trunc from 0x6b34a4 to 0x3d9d9a (0x2d9709 bytes) 000486[0] 1746136098.635549 mapread 0x13c000 thru 0x147208 (0xb209 bytes) 000488[1] 1746136098.671266 mapread 0x0a9000 thru 0x0aefb5 (0x5fb6 bytes) 000492[0] 1746136098.719927 trunc from 0x3d9d9b to 0x22c694 (0x1ad706 bytes) 000493[0] 1746136098.750292 read_OD 0x055000 thru 0x05efff (0xa000 bytes) 000496[0] 1746136098.773906 write_OD 0x148000 thru 0x149fff (0x2000 bytes) 000497[0] 1746136098.843344 write_OD 0x33a000 thru 0x348fff (0xf000 bytes) 000498[1] 1746136098.929784 write 0x052000 thru 0x0586b5 (0x66b6 bytes) 000500[0] 1746136098.948899 mapwrite 0x061000 thru 0x06775b (0x675c bytes) All operations completed A-OK! PASS 16a (42s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 17:48:22 (1746136102) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.450787 s, 18.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.33938 s, 24.7 MB/s Chance of close/open is 1 in 50 Seed set to 6104 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read skipping zero size read 000006[1] 1746136104.788383 mapwrite 0x75e19a thru 0x75e1f7 (0x5e bytes) 000008[1] 1746136105.008927 write 0x4360ac thru 0x437510 (0x1465 bytes) 000009[1] 1746136105.026286 read 0x24a207 thru 0x24b800 (0x15fa bytes) 000010[1] 1746136105.052337 mapwrite 0x4b5e0a thru 0x4b62e0 (0x4d7 bytes) 000011[1] 1746136105.130183 read 0x1c4d1f thru 0x1c6137 (0x1419 bytes) 000015[1] 1746136105.140182 read 0x6cfd3d thru 0x6d09ab (0xc6f bytes) truncating to largest ever: 0x94d41 000019[1] 1746136105.157840 trunc from 0x75e1f8 to 0x094d40 (0x6c94b7 bytes) 000020[1] 1746136105.178284 write 0x7e985d thru 0x7ea562 (0xd06 bytes) 000021[0] 1746136105.182558 mapwrite 0x0644a0 thru 0x064952 (0x4b3 bytes) truncating to largest ever: 0x1041db 000023[0] 1746136105.269337 trunc from 0x7ea563 to 0x1041da (0x6e6388 bytes) 000024[0] 1746136105.299093 read 0x06d4ab thru 0x06d641 (0x197 bytes) 000025[0] 1746136105.304853 read 0x0f31ee thru 0x0f3a15 (0x828 bytes) 000026[0] 1746136105.363190 mapread 0x0dfe06 thru 0x0e0ac6 (0xcc1 bytes) 000028[0] 1746136105.385163 mapread 0x033c22 thru 0x0345d6 (0x9b5 bytes) 000029[1] 1746136105.404621 mapwrite 0x8828b1 thru 0x882fa3 (0x6f3 bytes) truncating to largest ever: 0x4d6413 000032[1] 1746136105.535724 trunc from 0x882fa4 to 0x4d6412 (0x3acb91 bytes) 000034[0] 1746136105.597485 mapwrite 0x142e6f thru 0x143ac3 (0xc55 bytes) 000036[1] 1746136105.767905 trunc from 0x4d6413 to 0x18a47b (0x34bf97 bytes) 000038[1] 1746136105.813759 mapwrite 0x2cfb13 thru 0x2cfefb (0x3e9 bytes) 000039[1] 1746136105.988366 mapread 0x2bbf47 thru 0x2bdee1 (0x1f9b bytes) 000040[1] 1746136106.026908 mapread 0x267ff8 thru 0x269802 (0x180b bytes) 000043[0] 1746136106.112720 mapwrite 0x6065ad thru 0x607548 (0xf9c bytes) truncating to largest ever: 0x6e2527 000048[0] 1746136106.337817 trunc from 0x607549 to 0x6e2526 (0xdafde bytes) 000049[1] 1746136106.374291 read 0x66cf9f thru 0x66ee1b (0x1e7d bytes) 000052[1] 1746136106.460758 write 0x561c26 thru 0x56277f (0xb5a bytes) 000053[1] 1746136106.490573 write 0x192561 thru 0x192a84 (0x524 bytes) 000056[0] 1746136106.527450 read 0x57c212 thru 0x57c31c (0x10b bytes) 000059[0] 1746136106.582415 read 0x66cd89 thru 0x66d928 (0xba0 bytes) 000060[0] 1746136106.601000 trunc from 0x6e2527 to 0x573b1b (0x16ea0b bytes) 000061[0] 1746136106.652281 write 0x45437d thru 0x4556af (0x1333 bytes) 000062[0] 1746136106.681459 mapread 0x1d8ac0 thru 0x1d93f5 (0x936 bytes) 000063[1] 1746136106.710342 write 0x1396c4 thru 0x13b5c3 (0x1f00 bytes) 000066[1] 1746136106.787464 read 0x2be0eb thru 0x2be888 (0x79e bytes) 000068[1] 1746136106.814850 mapwrite 0x8050ac thru 0x8064a2 (0x13f7 bytes) 000069[0] 1746136106.972862 mapread 0x465d62 thru 0x4676b5 (0x1954 bytes) 000070[0] 1746136107.041276 read 0x0f2a34 thru 0x0f4863 (0x1e30 bytes) 000074[1] 1746136107.071261 mapwrite 0x88a5d4 thru 0x88c0f6 (0x1b23 bytes) 000075[1] 1746136107.213029 mapread 0x4b1a30 thru 0x4b2466 (0xa37 bytes) 000078[0] 1746136107.247259 write 0x827830 thru 0x82944e (0x1c1f bytes) 000080[1] 1746136107.282261 trunc from 0x88c0f7 to 0x6a8149 (0x1e3fad bytes) 000081[1] 1746136107.334668 read 0x094708 thru 0x095d9f (0x1698 bytes) 000086[0] 1746136107.412646 write 0x071d40 thru 0x072efb (0x11bc bytes) 000087[0] 1746136107.449788 mapread 0x6a46d1 thru 0x6a4e26 (0x756 bytes) truncating to largest ever: 0x908623 000089[0] 1746136107.478465 trunc from 0x6a814a to 0x908622 (0x2604d9 bytes) 000090[0] 1746136107.549281 read 0x84b07d thru 0x84c538 (0x14bc bytes) 000091[1] 1746136107.566181 trunc from 0x908623 to 0x5431f2 (0x3c5430 bytes) 000092[1] 1746136107.617281 read 0x1d0708 thru 0x1d1013 (0x90c bytes) 000093[1] 1746136107.620731 mapwrite 0x223cf0 thru 0x225ba4 (0x1eb5 bytes) 000096[1] 1746136107.767192 mapwrite 0x63b639 thru 0x63c8f1 (0x12b9 bytes) 000099[0] 1746136107.910121 mapread 0x311287 thru 0x311f09 (0xc83 bytes) 000100[0] 1746136107.960346 mapwrite 0x032c8f thru 0x0348ae (0x1c20 bytes) 000101[1] 1746136108.085881 mapwrite 0x675cd1 thru 0x6769ce (0xcfe bytes) 000102[1] 1746136108.252009 read 0x360978 thru 0x3618b5 (0xf3e bytes) 000107[0] 1746136108.308436 mapread 0x207349 thru 0x207d53 (0xa0b bytes) 000109[0] 1746136108.342704 write 0x25669a thru 0x257dca (0x1731 bytes) 000110[0] 1746136108.358708 write 0x48bc9d thru 0x48c295 (0x5f9 bytes) 000112[1] 1746136108.369245 write 0x8f8085 thru 0x8f9d3c (0x1cb8 bytes) 000113[0] 1746136108.395305 mapwrite 0x719830 thru 0x71a1ff (0x9d0 bytes) 000114[1] 1746136108.528014 trunc from 0x8f9d3d to 0x503451 (0x3f68eb bytes) 000117[0] 1746136108.580242 read 0x4aae68 thru 0x4ac816 (0x19af bytes) 000118[0] 1746136108.606634 mapwrite 0x23aaf9 thru 0x23bda2 (0x12aa bytes) 000120[1] 1746136108.705554 trunc from 0x503452 to 0x3c6a0d (0x13ca44 bytes) 000121[1] 1746136108.775020 read 0x1228d0 thru 0x123dd6 (0x1507 bytes) 000122[0] 1746136108.849080 mapread 0x16be96 thru 0x16dcd9 (0x1e44 bytes) 000123[0] 1746136108.908209 trunc from 0x3c6a0e to 0x45c7d9 (0x95dcc bytes) 000128[1] 1746136109.021454 mapwrite 0x15570c thru 0x15575f (0x54 bytes) 000132[0] 1746136109.196115 write 0x8ae664 thru 0x8af910 (0x12ad bytes) 000135[0] 1746136109.225680 mapread 0x673d42 thru 0x675b80 (0x1e3f bytes) 000136[1] 1746136109.250893 read 0x51fd27 thru 0x521376 (0x1650 bytes) 000137[1] 1746136109.302647 read 0x381fb4 thru 0x3831da (0x1227 bytes) 000139[1] 1746136109.320320 mapread 0x0f38cf thru 0x0f4d45 (0x1477 bytes) 000140[1] 1746136109.328508 read 0x4fc523 thru 0x4fdcce (0x17ac bytes) 000141[1] 1746136109.338769 trunc from 0x8af911 to 0x862fa1 (0x4c96f bytes) 000142[1] 1746136109.371360 mapwrite 0x4f148b thru 0x4f2d9d (0x1913 bytes) 000144[1] 1746136109.477836 mapread 0x3d3dab thru 0x3d4cb9 (0xf0f bytes) 000146[0] 1746136109.499124 mapread 0x5e199b thru 0x5e2c80 (0x12e6 bytes) 000147[0] 1746136109.548697 write 0x6516c1 thru 0x652822 (0x1162 bytes) 000148[0] 1746136109.583422 trunc from 0x862fa2 to 0x07d192 (0x7e5e0f bytes) 000152[0] 1746136109.675593 mapread 0x07b469 thru 0x07cf99 (0x1b31 bytes) 000155[1] 1746136109.744835 trunc from 0x07d193 to 0x0805ab (0x3419 bytes) 000156[0] 1746136109.797270 mapread 0x0129a5 thru 0x01338b (0x9e7 bytes) 000158[1] 1746136109.872587 trunc from 0x0805ac to 0x0b9739 (0x3918e bytes) 000159[1] 1746136109.919920 write 0x6b6d77 thru 0x6b806d (0x12f7 bytes) 000161[1] 1746136109.951841 trunc from 0x6b806e to 0x8348b7 (0x17c84a bytes) 000162[1] 1746136109.996119 read 0x749a71 thru 0x749d49 (0x2d9 bytes) 000166[1] 1746136110.035594 mapwrite 0x86c40c thru 0x86c9b7 (0x5ac bytes) 000170[0] 1746136110.169311 trunc from 0x86c9b8 to 0x5cbc71 (0x2a0d46 bytes) 000171[1] 1746136110.215415 write 0x61a3b6 thru 0x61a983 (0x5ce bytes) 000173[1] 1746136110.263042 write 0x74f6fc thru 0x74fa05 (0x30a bytes) 000174[1] 1746136110.287148 write 0x24856f thru 0x249e7c (0x190e bytes) 000175[1] 1746136110.322336 read 0x106ae6 thru 0x107779 (0xc94 bytes) 000178[1] 1746136110.327366 mapread 0x0d79a1 thru 0x0d7d20 (0x380 bytes) 000185[0] 1746136110.404456 mapread 0x3c73fa thru 0x3c7819 (0x420 bytes) 000186[0] 1746136110.436523 trunc from 0x74fa06 to 0x746604 (0x9401 bytes) 000188[0] 1746136110.521250 trunc from 0x746605 to 0x1764d0 (0x5d0134 bytes) 000190[0] 1746136110.582760 read 0x04ff56 thru 0x05154c (0x15f7 bytes) 000194[0] 1746136110.677282 write 0x57108e thru 0x5726a0 (0x1613 bytes) 000196[0] 1746136110.705252 write 0x196961 thru 0x196d49 (0x3e9 bytes) 000199[0] 1746136110.720729 trunc from 0x5726a1 to 0x3629c0 (0x20fce0 bytes) truncating to largest ever: 0x995442 000200[1] 1746136110.756316 trunc from 0x3629c1 to 0x995441 (0x632a81 bytes) 000203[1] 1746136110.831311 write 0x99b2a8 thru 0x99c6d8 (0x1431 bytes) 000206[1] 1746136110.850023 trunc from 0x99c6d9 to 0x28235b (0x71a37d bytes) 000208[0] 1746136110.868394 read 0x06bb43 thru 0x06bb4e (0xc bytes) 000211[0] 1746136110.892118 trunc from 0x28235c to 0x74d7fc (0x4cb4a1 bytes) 000214[0] 1746136110.969311 mapread 0x06bdde thru 0x06c119 (0x33c bytes) 000217[0] 1746136111.016569 mapwrite 0x23694e thru 0x237ba5 (0x1258 bytes) 000220[1] 1746136111.175466 mapwrite 0x1df9d8 thru 0x1e04be (0xae7 bytes) 000223[0] 1746136111.316965 write 0x0c986e thru 0x0cad15 (0x14a8 bytes) 000224[1] 1746136111.336869 mapwrite 0x420b32 thru 0x420e30 (0x2ff bytes) 000225[1] 1746136111.445377 trunc from 0x74d7fd to 0x36a690 (0x3e316c bytes) 000226[1] 1746136111.513322 trunc from 0x36a691 to 0x7f2c29 (0x488599 bytes) 000227[1] 1746136111.547233 write 0x97b1e4 thru 0x97d127 (0x1f44 bytes) 000230[0] 1746136111.583977 read 0x0f29ec thru 0x0f3dd3 (0x13e8 bytes) 000233[1] 1746136111.634415 read 0x34f0bb thru 0x34f195 (0xdb bytes) truncating to largest ever: 0x9ff180 000235[1] 1746136111.637717 trunc from 0x97d128 to 0x9ff17f (0x82058 bytes) 000237[0] 1746136111.685433 mapread 0x07ab8d thru 0x07c1c1 (0x1635 bytes) 000238[0] 1746136111.716550 read 0x580f70 thru 0x5828c6 (0x1957 bytes) 000239[1] 1746136111.740024 trunc from 0x9ff180 to 0x09f9e5 (0x95f79a bytes) 000240[0] 1746136111.796369 write 0x6dba38 thru 0x6dc66c (0xc35 bytes) 000243[1] 1746136111.860802 mapwrite 0x61f098 thru 0x61fe7d (0xde6 bytes) 000244[1] 1746136111.959547 write 0x0499d2 thru 0x04a333 (0x962 bytes) 000245[0] 1746136111.973838 write 0x34175c thru 0x341f80 (0x825 bytes) 000246[1] 1746136111.996872 write 0x61c2cf thru 0x61d87c (0x15ae bytes) 000247[0] 1746136112.031190 write 0x0fbfd4 thru 0x0fcc76 (0xca3 bytes) 000250[1] 1746136112.093876 mapread 0x05dc1a thru 0x05f429 (0x1810 bytes) 000251[0] 1746136112.126260 write 0x89f47d thru 0x8a1140 (0x1cc4 bytes) 000257[0] 1746136112.210190 mapwrite 0x485c9c thru 0x4868e1 (0xc46 bytes) 000259[1] 1746136112.333024 mapread 0x5ce0a9 thru 0x5cea27 (0x97f bytes) 000260[0] 1746136112.357735 mapread 0x0fc959 thru 0x0fd8db (0xf83 bytes) 000261[0] 1746136112.367669 mapread 0x374a74 thru 0x3751b1 (0x73e bytes) 000264[1] 1746136112.386838 trunc from 0x8a1141 to 0x9be02a (0x11ceea bytes) 000265[0] 1746136112.437243 mapwrite 0x5ec0e4 thru 0x5ed091 (0xfae bytes) 000267[1] 1746136112.543785 read 0x5f6414 thru 0x5f7ecb (0x1ab8 bytes) 000270[1] 1746136112.589415 trunc from 0x9be02b to 0x6c9106 (0x2f4f24 bytes) 000271[1] 1746136112.653120 read 0x0faaa9 thru 0x0fc6b2 (0x1c0a bytes) 000272[1] 1746136112.667305 mapread 0x19c5cc thru 0x19daa8 (0x14dd bytes) 000276[0] 1746136112.728412 trunc from 0x6c9107 to 0x52dfdd (0x19b129 bytes) 000277[0] 1746136112.763421 trunc from 0x52dfde to 0x0fdd40 (0x43029d bytes) 000278[0] 1746136112.783573 trunc from 0x0fdd41 to 0x77825d (0x67a51d bytes) 000281[1] 1746136112.800900 trunc from 0x77825e to 0x600a1f (0x17783e bytes) 000282[0] 1746136112.851150 read 0x299a4b thru 0x299b86 (0x13c bytes) 000284[1] 1746136112.898190 trunc from 0x600a20 to 0x0b28dd (0x54e142 bytes) 000286[0] 1746136112.961032 read 0x0671c3 thru 0x067984 (0x7c2 bytes) 000291[1] 1746136113.040333 trunc from 0x0b28de to 0x1c6f7d (0x1146a0 bytes) 000292[1] 1746136113.072263 mapwrite 0x31400f thru 0x314148 (0x13a bytes) 000295[0] 1746136113.190211 read 0x19a022 thru 0x19a79e (0x77d bytes) 000296[0] 1746136113.277338 trunc from 0x314149 to 0x297590 (0x7cbb8 bytes) 000297[0] 1746136113.338578 read 0x0ebaf5 thru 0x0ed0b8 (0x15c4 bytes) 000298[1] 1746136113.395424 write 0x02444c thru 0x0250ab (0xc60 bytes) 000302[0] 1746136113.478795 write 0x654fc9 thru 0x6560d1 (0x1109 bytes) 000303[1] 1746136113.504391 trunc from 0x6560d2 to 0x843b2e (0x1eda5d bytes) 000305[0] 1746136113.575117 mapread 0x61cba8 thru 0x61e955 (0x1dae bytes) 000307[0] 1746136113.649120 mapwrite 0x3a0bd0 thru 0x3a19e7 (0xe18 bytes) 000310[0] 1746136113.850714 read 0x129f85 thru 0x12a331 (0x3ad bytes) 000314[0] 1746136113.885798 read 0x123b65 thru 0x125392 (0x182e bytes) 000319[0] 1746136113.924212 write 0x12d614 thru 0x12ea9a (0x1487 bytes) 000330[1] 1746136114.065156 mapwrite 0x219249 thru 0x21aa0c (0x17c4 bytes) 000331[0] 1746136114.175919 write 0x0e30f4 thru 0x0e3723 (0x630 bytes) 000334[1] 1746136114.205748 read 0x13e16a thru 0x13f055 (0xeec bytes) 000338[0] 1746136114.229406 mapwrite 0x9921c3 thru 0x992db4 (0xbf2 bytes) 000339[0] 1746136114.348786 mapread 0x180101 thru 0x180354 (0x254 bytes) 000344[0] 1746136114.382647 mapwrite 0x30cc12 thru 0x30da84 (0xe73 bytes) 000345[1] 1746136114.467473 mapread 0x630fc3 thru 0x63113d (0x17b bytes) 000346[1] 1746136114.517980 trunc from 0x992db5 to 0x43f299 (0x553b1b bytes) 000350[1] 1746136114.593150 mapwrite 0x582eb0 thru 0x584355 (0x14a6 bytes) 000351[1] 1746136114.690713 mapread 0x2f9888 thru 0x2f9b62 (0x2db bytes) 000352[0] 1746136114.697852 read 0x08a317 thru 0x08a59a (0x284 bytes) 000359[1] 1746136114.792187 write 0x7f9190 thru 0x7f9b1e (0x98f bytes) 000363[0] 1746136114.850882 read 0x40479b thru 0x405bec (0x1452 bytes) 000364[1] 1746136114.897175 mapread 0x3807ce thru 0x380822 (0x55 bytes) 000365[0] 1746136114.903068 read 0x7b1141 thru 0x7b2c36 (0x1af6 bytes) 000366[0] 1746136114.910624 mapwrite 0x2fcf6c thru 0x2fe138 (0x11cd bytes) 000367[1] 1746136115.035484 trunc from 0x7f9b1f to 0x242ec6 (0x5b6c58 bytes) 000369[0] 1746136115.092236 read 0x104078 thru 0x105f28 (0x1eb1 bytes) 000371[1] 1746136115.153215 read 0x18c486 thru 0x18d4af (0x102a bytes) 000373[0] 1746136115.229338 trunc from 0x242ec7 to 0x081a21 (0x1c14a5 bytes) 000374[0] 1746136115.264468 mapwrite 0x8d8a1a thru 0x8d92e9 (0x8d0 bytes) 000375[1] 1746136115.423802 trunc from 0x8d92ea to 0x131229 (0x7a80c0 bytes) 000376[0] 1746136115.469066 mapread 0x0eb0e4 thru 0x0eb507 (0x424 bytes) 000377[0] 1746136115.502391 write 0x8a0620 thru 0x8a20b8 (0x1a99 bytes) 000378[0] 1746136115.524763 write 0x9ec4c6 thru 0x9edd99 (0x18d4 bytes) 000379[0] 1746136115.547840 mapwrite 0x25262f thru 0x253641 (0x1013 bytes) 000381[1] 1746136115.676711 trunc from 0x9edd9a to 0x34758c (0x6a680d bytes) 000383[1] 1746136115.729968 trunc from 0x34758d to 0x7cc3a1 (0x484e15 bytes) 000385[1] 1746136115.772823 read 0x1aa61a thru 0x1abe6c (0x1853 bytes) 000390[0] 1746136115.816313 mapwrite 0x666627 thru 0x667d7e (0x1758 bytes) 000392[1] 1746136115.963124 mapwrite 0x478d70 thru 0x47a98c (0x1c1d bytes) 000393[0] 1746136116.090159 mapwrite 0x837c6a thru 0x839950 (0x1ce7 bytes) 000399[1] 1746136116.244096 read 0x69a8b1 thru 0x69c585 (0x1cd5 bytes) 000400[1] 1746136116.264282 read 0x421ce7 thru 0x4225c1 (0x8db bytes) 000401[1] 1746136116.268766 write 0x2a04c1 thru 0x2a1948 (0x1488 bytes) 000403[0] 1746136116.309194 read 0x6590c3 thru 0x659cb2 (0xbf0 bytes) 000405[1] 1746136116.330878 write 0x12e301 thru 0x130193 (0x1e93 bytes) 000406[0] 1746136116.352211 mapwrite 0x67a605 thru 0x67b5ee (0xfea bytes) 000410[0] 1746136116.500429 trunc from 0x839951 to 0x160afb (0x6d8e55 bytes) 000413[1] 1746136116.527718 mapwrite 0x6feaf4 thru 0x6ffe60 (0x136d bytes) 000414[0] 1746136116.647428 trunc from 0x6ffe61 to 0x4434ff (0x2bc961 bytes) 000416[1] 1746136116.687702 write 0x35ad72 thru 0x35c28b (0x151a bytes) 000418[0] 1746136116.728440 trunc from 0x443500 to 0x3e7f3a (0x5b5c5 bytes) 000420[0] 1746136116.771824 mapread 0x2f0b69 thru 0x2f28e9 (0x1d81 bytes) 000423[0] 1746136116.790263 trunc from 0x3e7f3b to 0x2bb372 (0x12cbc8 bytes) 000426[1] 1746136116.839801 trunc from 0x2bb373 to 0x5d199a (0x316628 bytes) 000431[1] 1746136116.941997 write 0x23c9c6 thru 0x23e08a (0x16c5 bytes) 000433[0] 1746136116.962172 mapread 0x4e90d9 thru 0x4e9823 (0x74b bytes) 000436[0] 1746136117.002549 mapread 0x472be8 thru 0x4737ed (0xc06 bytes) 000438[1] 1746136117.041058 read 0x15b0a2 thru 0x15b7f5 (0x754 bytes) 000439[1] 1746136117.065394 mapread 0x147e2a thru 0x148534 (0x70b bytes) 000441[1] 1746136117.081775 write 0x9080e2 thru 0x908d8b (0xcaa bytes) 000443[1] 1746136117.096884 mapwrite 0x7ff417 thru 0x801274 (0x1e5e bytes) 000444[0] 1746136117.168417 mapread 0x6cfef7 thru 0x6d09c9 (0xad3 bytes) 000452[1] 1746136117.245549 write 0x81de86 thru 0x81fe01 (0x1f7c bytes) 000453[1] 1746136117.262011 mapwrite 0x8b704d thru 0x8b8072 (0x1026 bytes) 000457[0] 1746136117.370333 write 0x3d224c thru 0x3d34cf (0x1284 bytes) 000458[0] 1746136117.392799 write 0x0aeba3 thru 0x0aedca (0x228 bytes) 000459[1] 1746136117.396279 mapwrite 0x221233 thru 0x221aa3 (0x871 bytes) 000465[0] 1746136117.535723 trunc from 0x908d8c to 0x84ee1a (0xb9f71 bytes) 000467[0] 1746136117.588224 mapread 0x3fb5d1 thru 0x3fc042 (0xa72 bytes) 000470[1] 1746136117.631139 write 0x7270cc thru 0x728516 (0x144b bytes) 000472[0] 1746136117.676948 mapwrite 0x496348 thru 0x496ca5 (0x95e bytes) 000474[0] 1746136117.760620 write 0x03ce53 thru 0x03decf (0x107d bytes) 000475[1] 1746136117.769690 read 0x07b016 thru 0x07cb89 (0x1b74 bytes) 000477[1] 1746136117.807225 read 0x712ff8 thru 0x71458c (0x1595 bytes) 000478[1] 1746136117.829619 trunc from 0x84ee1b to 0x7efd10 (0x5f10a bytes) 000479[0] 1746136117.875281 trunc from 0x7efd11 to 0x6cd204 (0x122b0c bytes) 000481[1] 1746136117.930337 trunc from 0x6cd205 to 0x4e0c61 (0x1ec5a3 bytes) 000488[0] 1746136118.075542 mapwrite 0x8fde10 thru 0x8ffdbb (0x1fac bytes) 000489[0] 1746136118.215344 mapwrite 0x5103d4 thru 0x51053a (0x167 bytes) 000490[0] 1746136118.301891 mapwrite 0x52eb10 thru 0x530953 (0x1e44 bytes) 000491[1] 1746136118.393552 mapread 0x74f8bc thru 0x750ddc (0x1521 bytes) 000492[0] 1746136118.447436 read 0x2c8de5 thru 0x2c8ecf (0xeb bytes) 000493[0] 1746136118.468736 write 0x5f21ee thru 0x5f2266 (0x79 bytes) 000495[1] 1746136118.513456 mapwrite 0x5daf05 thru 0x5dba1a (0xb16 bytes) 000496[0] 1746136118.620511 write 0x2765d1 thru 0x2778fb (0x132b bytes) 000497[0] 1746136118.654222 write 0x34ff10 thru 0x350259 (0x34a bytes) 000500[0] 1746136118.744401 write 0x89d7d2 thru 0x89e07a (0x8a9 bytes) All operations completed A-OK! PASS 16b (19s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 17:48:41 (1746136121) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 17:48:44 (1746136124) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.455213 s, 18.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.363326 s, 23.1 MB/s ldlm.namespaces.MGC192.168.201.121@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa034800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800af65b000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa034800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800af65b000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa034800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800af65b000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76504 s, 10.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.18953 s, 47.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.50031 s, 69.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.36703 s, 24.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 8.08922 s, 13.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.27919 s, 46.0 MB/s PASS 16d (37s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 17:49:21 (1746136161) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.193486 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154546 s, 6.8 MB/s PASS 16e (6s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 17:49:27 (1746136167) TIMEOUT PASS 16f (25s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 17:49:52 (1746136192) TIMEOUT PASS 16g (25s) == sanityn test 16h: mmap read after truncate file ======= 17:50:17 (1746136217) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (6s) == sanityn test 16i: read after truncate file ============ 17:50:23 (1746136223) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (4s) == sanityn test 16j: race dio with buffered i/o ========== 17:50:27 (1746136227) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 1.04846 s, 25.0 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.987205 s, 26.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 2.54714 s, 10.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.457389 s, 57.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 2.00857 s, 13.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 dd: error reading '/mnt/lustre/f16j.sanityn.1': Invalid argument 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.134853 s, 0.0 kB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.482625 s, 54.3 MB/s sanityn test_16j: @@@@@@ FAIL: dio copy 1 w/bsize 1049600 failed: 1 Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7249:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:760:test_16j() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7622:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7685:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7503:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:830:main() Dumping lctl log to /tmp/testlogs//sanityn.test_16j.*.1746136239.log rsync: chown "/tmp/testlogs/.sanityn.test_16j.debug_log.oleg121-server.1746136239.log.PWgt41" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_16j.dmesg.oleg121-server.1746136239.log.87Ekqn" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2] FAIL 16j (16s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 17:50:47 (1746136247) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x3c569db truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Thu May 1 17:51:21 EDT 2025 + fsx 2 done: rc=0 Thu May 1 17:51:21 EDT 2025 + fsx 3 done: rc=0 Thu May 1 17:51:21 EDT 2025 All operations completed A-OK! + fsx 4 done: rc=0 Thu May 1 17:51:22 EDT 2025 + fsx 5 done: rc=0 Thu May 1 17:51:22 EDT 2025 PASS 16k (41s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 17:51:29 (1746136289) fail_loc=0x8000030a PASS 17 (7s) == sanityn test 18: mmap sanity check =========================================================================================== 17:51:36 (1746136296) mmap test1: basic mmap operation (PASS, 0.10286s) mmap test2: MAP_PRIVATE not write back (PASS, 0.056396s) mmap test3: concurrent mmap ops on two nodes (PASS, 19.011s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.3626s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.11692s) mmap test6: check mmap write/read content on two nodes (PASS, 0.26959s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.93546s) mmap test11: multi-thread shared mmap (PASS, 0.78833s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0799s) PASS 18 (33s) == sanityn test 19: test concurrent uncached read races ========================================================================= 17:52:09 (1746136329) oleg121-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg121-client: oleg121-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (3s) == sanityn test 20: test extra readahead page left in cache ============================================================== 17:52:12 (1746136332) PASS 20 (5s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 17:52:17 (1746136337) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 17:52:21 (1746136341) now is 1746136343 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 new atime is 1746136404 PASS 23 (67s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 17:53:28 (1746136408) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre2[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 114.9K 406 114.5K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.1K 403 113.7K 1% /mnt/lustre[OST:1] filesystem_summary: 228.6K 387 228.2K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 114.9K 406 114.5K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.1K 403 113.7K 1% /mnt/lustre2[OST:1] filesystem_summary: 228.6K 387 228.2K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 19.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 44.0M 3.6G 2% /mnt/lustre[OST:1] filesystem_summary: 7.2G 63.0M 7.1G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117654 406 117248 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 116851 403 116448 1% /mnt/lustre2[OST:1] filesystem_summary: 234083 387 233696 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 114.9K 406 114.5K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.1K 403 113.7K 1% /mnt/lustre2[OST:1] filesystem_summary: 228.6K 387 228.2K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117654 406 117248 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 116851 403 116448 1% /mnt/lustre2[OST:1] filesystem_summary: 234083 387 233696 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre2[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre2 PASS 24a (4s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 17:53:32 (1746136412) PASS 24b (4s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 17:53:37 (1746136417) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (5s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 17:53:41 (1746136421) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (2s) == sanityn test 26a: allow mtime to get older ============ 17:53:44 (1746136424) utime: good mknod times 1746136424 <= 1746136425 <= 1746136425 for /mnt/lustre/f26a utime: good mknod times 1746136424 <= 1746136425 <= 1746136425 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 17:53:49 (1746136429) PASS 26b (6s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 17:53:55 (1746136435) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502391 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-05-01 17:53:56.000000000 -0400 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502391 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-05-01 17:53:56.000000000 -0400 Birth: - PASS 26c (6s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 17:54:03 (1746136443) 0 PASS 30 (6s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 17:54:09 (1746136449) fail_loc=0x314 PASS 31a (5s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 17:54:15 (1746136455) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg121-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.121@tcp:/lustre 7542784 64512 7470080 1% /mnt/lustre2 PASS 31b (16s) == sanityn test 31r: open-rename(replace) race =========== 17:54:31 (1746136471) -rw-r--r-- 1 root root 0 May 1 17:54 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (8s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 17:54:39 (1746136479) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13755 Terminated tail -f $tfile PASS 31s (6s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 17:54:45 (1746136485) PASS 31t (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 17:54:51 (1746136491) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 17:54:53 (1746136493) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 17:54:55 (1746136495) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 17:54:57 (1746136497) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 17:54:59 (1746136499) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa034800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa034800.ost_server_uuid in IDLE state after 0 sec oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa034800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aa034800.ost_server_uuid in FULL state after 0 sec oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa034800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa034800.ost_server_uuid in IDLE state after 0 sec oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa034800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aa034800.ost_server_uuid in FULL state after 0 sec oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa034800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa034800.ost_server_uuid in IDLE state after 0 sec oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa034800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aa034800.ost_server_uuid in FULL state after 0 sec PASS 34 (107s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 17:56:45 (1746136605) fail_loc=0x80000317 Race attempt 0 Wait for 20420 20490 for 60 sec... - open/close 857 (time 1746136620.89 total 10.02 last 85.49) - open/close 1838 (time 1746136630.89 total 20.03 last 98.04) - open/close 2859 (time 1746136640.90 total 30.04 last 102.02) - open/close 3882 (time 1746136650.90 total 40.04 last 102.30) total: 4000 open/close in 41.11 seconds: 97.30 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.121@tcp:/lustre 7.2G 64M 7.2G 1% /mnt/lustre 192.168.201.121@tcp:/lustre 7.2G 64M 7.2G 1% /mnt/lustre2 PASS 35 (69s) == sanityn test 36: handle ESTALE/open-unlink correctly == 17:57:54 (1746136674) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg121-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.951842 s, 55.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7473152) after_dd(7417856) after(7469056) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.800855 s, 65.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7469056) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.730122 s, 71.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7467008) after_dd(7412736) after(7463936) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.671385 s, 78.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(3) *** before(7463936) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.801607 s, 65.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(4) *** before(7467008) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.734413 s, 71.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7467008) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.758393 s, 69.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7467008) after_dd(7417856) after(7469056) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.755346 s, 69.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7469056) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.836328 s, 62.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7467008) after_dd(7416832) after(7468032) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.589653 s, 88.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7468032) after_dd(7412736) after(7463936) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.715357 s, 73.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7463936) after_dd(7415808) after(7467008) PASS 36 (245s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 18:02:00 (1746136920) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 - create 1895 (time 1746136933.20 total 10.01 last 189.34) - create 3580 (time 1746136943.20 total 20.01 last 168.49) - create 5213 (time 1746136953.20 total 30.01 last 163.29) - create 6773 (time 1746136963.20 total 40.01 last 155.95) - create 8480 (time 1746136973.21 total 50.02 last 170.61) total: 10000 create in 58.21 seconds: 171.80 ops/second PASS 37 (65s) == sanityn test 39a: file mtime does not change after rename ========================================================== 18:03:06 (1746136986) repeat after cancel_lru_locks PASS 39a (5s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 18:03:11 (1746136991) repeat after cancel_lru_locks PASS 39b (6s) == sanityn test 39c: check truncate mtime update ================================================================================ 18:03:17 (1746136997) repeat after cancel_lru_locks PASS 39c (6s) == sanityn test 39d: sync write should update mtime ====== 18:03:23 (1746137003) fail_loc=0x411 fail_loc=0 PASS 39d (5s) == sanityn test 40a: pdirops: create vs others ======================================================================== 18:03:28 (1746137008) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (14s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 18:03:43 (1746137023) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (15s) == sanityn test 40c: pdirops: link and others ======================================================================== 18:03:57 (1746137037) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (15s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 18:04:12 (1746137052) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (13s) == sanityn test 40e: pdirops: rename and others ======================================================================== 18:04:25 (1746137065) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (12s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 18:04:38 (1746137078) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (9s) == sanityn test 41b: pdirops: create vs create ======================================================================== 18:04:47 (1746137087) open: File exists Conflict PASS 41b (10s) == sanityn test 41c: pdirops: create vs link ======================================================================== 18:04:58 (1746137098) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (12s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 18:05:09 (1746137109) Conflict PASS 41d (10s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 18:05:19 (1746137119) Conflict PASS 41e (12s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 18:05:31 (1746137131) Conflict PASS 41f (11s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 18:05:42 (1746137142) Conflict PASS 41g (10s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 18:05:52 (1746137152) Conflict PASS 41h (9s) == sanityn test 41i: reint_open: create vs create ======== 18:06:01 (1746137161) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (916s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 18:21:17 (1746138077) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (9s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 18:21:26 (1746138086) open: File exists Conflict PASS 42b (9s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 18:21:35 (1746138095) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (10s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 18:21:45 (1746138105) Conflict PASS 42d (9s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 18:21:54 (1746138114) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (11s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 18:22:05 (1746138125) Conflict PASS 42f (11s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 18:22:16 (1746138136) Conflict PASS 42g (8s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 18:22:25 (1746138145) Conflict PASS 42h (9s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 18:22:34 (1746138154) PASS 43a (45s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 18:23:19 (1746138199) Conflict PASS 43b (9s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 18:23:28 (1746138208) Conflict PASS 43c (10s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 18:23:38 (1746138218) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (9s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 18:23:47 (1746138227) Conflict PASS 43e (10s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 18:23:58 (1746138238) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (10s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 18:24:07 (1746138247) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (10s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 18:24:18 (1746138258) Conflict PASS 43h (9s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 18:24:28 (1746138268) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (2s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 18:24:30 (1746138270) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (109s) == sanityn test 43k: unlink vs create ==================== 18:26:19 (1746138379) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (952s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 18:42:11 (1746139331) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu May 1 18:42:18 EDT 2025 PASS 44a (10s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 18:42:21 (1746139341) open: File exists Conflict PASS 44b (9s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 18:42:30 (1746139350) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (8s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 18:42:39 (1746139359) Conflict PASS 44d (9s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 18:42:47 (1746139367) Conflict PASS 44e (10s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 18:42:57 (1746139377) Conflict PASS 44f (9s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 18:43:06 (1746139386) Conflict PASS 44g (9s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 18:43:15 (1746139395) Conflict PASS 44h (11s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 18:43:26 (1746139406) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (2s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 18:43:28 (1746139408) PASS 45a (98s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 18:45:06 (1746139506) Conflict PASS 45b (11s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 18:45:18 (1746139518) Conflict PASS 45c (12s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 18:45:29 (1746139529) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (11s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 18:45:40 (1746139540) Conflict PASS 45e (11s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 18:45:51 (1746139551) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (11s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 18:46:02 (1746139562) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (11s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 18:46:13 (1746139573) Conflict PASS 45h (10s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 18:46:23 (1746139583) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (2s) == sanityn test 45j: read vs rename ====================== 18:46:25 (1746139585) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (873s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 19:00:58 (1746140458) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (10s) == sanityn test 46b: pdirops: link vs create ======================================================================== 19:01:08 (1746140468) open: File exists Conflict PASS 46b (11s) == sanityn test 46c: pdirops: link vs link ======================================================================== 19:01:19 (1746140479) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (10s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 19:01:30 (1746140490) Conflict PASS 46d (11s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 19:01:40 (1746140500) Conflict PASS 46e (9s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 19:01:49 (1746140509) Conflict PASS 46f (10s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 19:01:59 (1746140519) Conflict PASS 46g (10s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 19:02:09 (1746140529) Conflict PASS 46h (12s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 19:02:21 (1746140541) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (2s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 19:02:23 (1746140543) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 19:02:26 (1746140546) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (2s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 19:02:28 (1746140548) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (3s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 19:02:31 (1746140551) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 19:02:33 (1746140553) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (2s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 19:02:36 (1746140556) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 19:02:38 (1746140558) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (2s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 19:02:40 (1746140560) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.178662 s, 57.3 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (10s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 19:02:50 (1746140570) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.0414497 s, 3.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3519: kill: (11825) - No such process PASS 51a (7s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 19:02:57 (1746140577) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.014354 s, 71.3 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498307 s, 21.0 MB/s 3145728 PASS 51b (17s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 19:03:14 (1746140594) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0197912 s, 51.7 kB/s PASS 51c (10s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 19:03:24 (1746140604) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552284 s, 19.0 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (6s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 19:03:30 (1746140610) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2444 0x98c 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2444 0x98c 0x280000400 read lease(1) has applied. PASS 51e (7s) == sanityn test 54: rename locking ======================= 19:03:37 (1746140617) llite.lustre-ffff8800aa034800.xattr_cache=1 llite.lustre-ffff8800af65b000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505005 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:03:38.000000000 -0400 Modify: 2025-05-01 19:03:38.000000000 -0400 Change: 2025-05-01 19:03:38.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505008 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:03:44.000000000 -0400 Modify: 2025-05-01 19:03:44.000000000 -0400 Change: 2025-05-01 19:03:44.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:03:49.000000000 -0400 Modify: 2025-05-01 19:03:49.000000000 -0400 Change: 2025-05-01 19:03:49.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505010 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:03:49.000000000 -0400 Modify: 2025-05-01 19:03:50.000000000 -0400 Change: 2025-05-01 19:03:50.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505014 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:03:55.000000000 -0400 Modify: 2025-05-01 19:03:56.000000000 -0400 Change: 2025-05-01 19:03:56.000000000 -0400 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505013 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:03:55.000000000 -0400 Modify: 2025-05-01 19:03:56.000000000 -0400 Change: 2025-05-01 19:03:56.000000000 -0400 Birth: - PASS 54 (28s) == sanityn test 55a: rename vs unlink target dir ========= 19:04:05 (1746140645) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (10s) == sanityn test 55b: rename vs unlink source dir ========= 19:04:16 (1746140656) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (10s) == sanityn test 55c: rename vs unlink orphan target dir == 19:04:26 (1746140666) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (16s) == sanityn test 55d: rename file vs link ================= 19:04:42 (1746140682) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (13s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 19:04:55 (1746140695) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1746140696 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 0.813854 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (80s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 19:06:15 (1746140775) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1746140778 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 1.50709 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (91s) == sanityn test 60: Verify data_version behaviour ======== 19:07:46 (1746140866) PASS 60 (5s) == sanityn test 70a: cd directory && rm directory ======== 19:07:52 (1746140872) PASS 70a (6s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 19:07:58 (1746140878) PASS 70b (7s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 19:08:06 (1746140886) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00991611 s, 6.6 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 19:08:09 (1746140889) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0132491 s, 4.9 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (3s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 19:08:12 (1746140892) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 19:08:15 (1746140895) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0173071 s, 3.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (3s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 19:08:18 (1746140898) llite.lustre-ffff8800aa034800.xattr_cache=1 llite.lustre-ffff8800af65b000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (5s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 19:08:23 (1746140903) llite.lustre-ffff8800aa034800.xattr_cache=1 llite.lustre-ffff8800af65b000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (5s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 19:08:28 (1746140908) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0123203 s, 83.1 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00746 s, 137 kB/s 24400: get lock2 24401: get lock1 24400: done 24400: get lock1 24401: done 24401: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 24401: cannot get lock2: Resource deadlock avoided 24401: exit rc=0 24400: done 24400: exit rc=0 PASS 74 (8s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 19:08:36 (1746140916) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.030463 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034363 s, 30.5 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4108: 25045 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (11s) == sanityn test 76: Verify MDT open_files listing ======== 19:08:47 (1746140927) opening files: ........ closing files: ........ PASS 76 (51s) == sanityn test 77a: check FIFO NRS policy =============== 19:09:38 (1746140978) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.227042 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014659 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352869 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391774 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430159 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415661 s, 25.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971399 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0467691 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645288 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0984318 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10172 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140471 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164111 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165447 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116765 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151019 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.195299 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0699446 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361858 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0685407 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0739327 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042322 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0621722 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351353 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379396 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0724651 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398521 s, 26.3 MB/s 1048576 bytes (1.0 MB) copied, 0.042704 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443829 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464749 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0509844 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515538 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0559171 s, 18.8 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (6s) == sanityn test 77b: check CRR-N NRS policy ============== 19:09:44 (1746140984) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.330758 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313664 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401946 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254056 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0935146 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0980514 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0972375 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131023 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116563 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133085 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137844 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.119723 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149246 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170462 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169085 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177494 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.195429 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536946 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00202268 s, 518 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00246404 s, 426 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00242398 s, 433 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00662912 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0996777 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00212347 s, 494 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0869498 s, 12.1 MB/s , 0.102853 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0875484 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.090976 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0937992 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0876268 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824458 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565162 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0912285 s, 11.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.295846 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042104 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498227 s, 21.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108017 s, 9.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0990732 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110749 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110034 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13998 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113305 s, 9.3 MB/s 1+0 records in 1+0 records out , 0.145112 s, 7.2 MB/s 1048576 bytes (1.0 MB) copied, 0.128971 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151595 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144155 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1491 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185211 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.204136 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.228479 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516055 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364763 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534823 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542209 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371162 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0675718 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0692984 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674346 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448376 s, 23.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0473942 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463595 s, 22.6 MB/s , 0.0375845 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477482 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483877 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535716 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620042 s, 16.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (10s) == sanityn test 77c: check ORR NRS policy ================ 19:09:54 (1746140994) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.251976 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436167 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359426 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124772 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122875 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128344 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122858 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106171 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129895 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115228 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141732 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155893 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132196 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12503 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146497 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143328 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17087 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0302418 s, 34.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00289348 s, 362 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257243 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290322 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027778 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.028993 s, 36.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00214161 s, 490 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362747 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00336791 s, 311 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00435768 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327312 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447048 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0164517 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00172873 s, 607 MB/s , 0.025194 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436187 s, 24.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.2017 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227504 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246864 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0727936 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0827265 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0945787 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0928127 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0997632 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106782 s, 9.8 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103516 s, 10.1 MB/s , 0.107452 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0985199 s, 10.6 MB/s 1+0 records in 1+0 records out , 0.105394 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106988 s, 9.8 MB/s 1048576 bytes (1.0 MB) copied, 0.119335 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118279 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116294 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210027 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315225 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0019466 s, 539 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00567486 s, 185 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0212209 s, 49.4 MB/s , 0.00323071 s, 325 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00250858 s, 418 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638442 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570616 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00181139 s, 579 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0403942 s, 26.0 MB/s , 0.0541941 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259657 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235693 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022877 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182853 s, 57.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.170749 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139934 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019391 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0555036 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0586898 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0599898 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0802217 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813141 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0840845 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0802947 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0821603 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0893321 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0969415 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0976492 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0917482 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0995194 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105358 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250662 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00246951 s, 425 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255291 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027258 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00527428 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176269 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255929 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264428 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00172181 s, 609 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165141 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350375 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0041496 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00265584 s, 395 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00281613 s, 372 MB/s , 0.0112644 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00705218 s, 149 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (8s) == sanityn test 77d: check TRR nrs policy ================ 19:10:02 (1746141002) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.245137 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164406 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326497 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459718 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534558 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393121 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659541 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.077447 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638018 s, 16.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0679603 s, 15.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0939623 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0736763 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0996117 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104076 s, 10.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0951275 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0950496 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0797539 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00575214 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00490582 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00667534 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0860641 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00304687 s, 344 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662662 s, 15.8 MB/s , 0.0709843 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0761357 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0877027 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0889457 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596277 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489505 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463612 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343824 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285477 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470334 s, 22.3 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.163676 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260964 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505908 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0904555 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0979687 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103311 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114654 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108601 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114318 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123308 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137159 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145955 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158285 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153446 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167848 s, 6.2 MB/s 1048576 bytes (1.0 MB) copied, 0.176437 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187147 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0454153 s, 23.1 MB/s , 0.0436658 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508926 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505065 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520493 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528449 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0543465 s, 19.3 MB/s , 0.0551654 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0530402 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570218 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.064756 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399447 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397422 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407521 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040609 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0738298 s, 14.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.177737 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349718 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321168 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0846425 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0941541 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0991287 s, 10.6 MB/s 1048576 bytes (1.0 MB) copied, 0.092999 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0971773 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1163 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11312 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113124 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115098 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108551 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111305 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122405 s, 8.6 MB/s 1048576 bytes (1.0 MB) copied, 0.120439 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135816 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310954 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305805 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232738 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00407394 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260261 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00479002 s, 219 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00287787 s, 364 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286126 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055982 s, 18.7 MB/s , 0.00960806 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.060756 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00662095 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040689 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493923 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0034852 s, 301 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380265 s, 27.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (9s) == sanityn test 77e: check TBF NID nrs policy ============ 19:10:11 (1746141011) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.21@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.21507 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347638 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0533212 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0816817 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0966777 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0981105 s, 10.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0948359 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120745 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129123 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110276 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126812 s, 8.3 MB/s , 0.0964515 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130919 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121963 s, 8.6 MB/s 1048576 bytes (1.0 MB) copied, 0.141025 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128131 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141986 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224126 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019619 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0044023 s, 238 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00207684 s, 505 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0737658 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00550561 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0723541 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00676912 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118501 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11225 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0807066 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0641052 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0958525 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501577 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485445 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405818 s, 25.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.197756 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209115 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269033 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294514 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286475 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289628 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133181 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115951 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118263 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127326 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116948 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146941 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15119 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153291 s, 6.8 MB/s , 0.148717 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141271 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162498 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111813 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00279248 s, 375 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162658 s, 64.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.048154 s, 21.8 MB/s , 0.06018 s, 17.4 MB/s , 0.0445655 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602792 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404497 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405754 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435438 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918126 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460656 s, 22.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0189849 s, 55.2 MB/s , 0.0500775 s, 20.9 MB/s , 0.0282216 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364372 s, 28.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.309181 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293559 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404201 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0740931 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104354 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103493 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0860379 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122244 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121435 s, 8.6 MB/s 1048576 bytes (1.0 MB) copied, 0.121361 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124833 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11842 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129216 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109512 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10812 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153776 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156205 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0215427 s, 48.7 MB/s , 0.0227287 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234526 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291886 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259554 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209097 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276358 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221138 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243611 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245641 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243585 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026737 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0303592 s, 34.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0304229 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031168 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372276 s, 28.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.225965 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340156 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334254 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465449 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180817 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051199 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287119 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496556 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.097104 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149866 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137193 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148906 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121087 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120969 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152655 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1675 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.214944 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00241739 s, 434 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481175 s, 21.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0116188 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405812 s, 25.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00360534 s, 291 MB/s 1048576 bytes (1.0 MB) copied, 0.0388403 s, 27.0 MB/s , 0.033841 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0701508 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00579131 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00283202 s, 370 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0676666 s, 15.5 MB/s , 0.0460152 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482012 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211604 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505595 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189904 s, 55.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (14s) == sanityn test 77f: check TBF JobID nrs policy ========== 19:10:25 (1746141025) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.256676 s, 65.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242135 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233041 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241455 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0781203 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0794884 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0677674 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0959659 s, 10.9 MB/s 1048576 bytes (1.0 MB) copied, 0.105719 s, 9.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0960767 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115233 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10706 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112924 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0946954 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107425 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117192 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116166 s, 9.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178066 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183586 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177019 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17685 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188325 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172319 s, 6.1 MB/s 1048576 bytes (1.0 MB) copied, 0.190091 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17334 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.189444 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184593 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174235 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1706 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170948 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187859 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181675 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.189184 s, 5.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.155916 s, 108 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253184 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277026 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167347 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294843 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142444 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0864154 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0998962 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0923594 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100904 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105689 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1111 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107499 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112275 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121674 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125938 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133952 s, 7.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301537 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365169 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317192 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364115 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406279 s, 25.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035288 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362175 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372128 s, 28.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0440478 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374678 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344271 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426072 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372063 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371341 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392391 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453774 s, 23.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.194792 s, 86.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252199 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281924 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217058 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165137 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291906 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326151 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274554 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125016 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0971587 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.088255 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133351 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0992214 s, 10.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131334 s, 8.0 MB/s , 0.0947487 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104092 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147442 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00536631 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143011 s, 73.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0230624 s, 45.5 MB/s , 0.0310183 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174729 s, 60.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0141285 s, 74.2 MB/s , 0.01431 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189357 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159947 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174641 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180598 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210957 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225392 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228358 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229624 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024164 s, 43.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.187819 s, 89.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986109 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196325 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0282087 s, 37.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0290613 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172494 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0920382 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104674 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0839136 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128715 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131869 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122456 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117256 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130477 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131422 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139317 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148911 s, 7.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0209395 s, 50.1 MB/s 1048576 bytes (1.0 MB) copied, 0.030366 s, 34.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0323522 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247786 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024871 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274892 s, 38.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0290688 s, 36.1 MB/s , 0.0288943 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307538 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313391 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328917 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357255 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345127 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372659 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395054 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429878 s, 24.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (13s) == sanityn test 77g: Change TBF type directly ============ 19:10:38 (1746141038) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (7s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 19:10:45 (1746141045) oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg121-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg121-client: oleg121-server: ssh exited with exit code 2 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg121-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg121-client: oleg121-server: ssh exited with exit code 12 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg121-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg121-client: oleg121-server: ssh exited with exit code 12 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg121-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg121-client: oleg121-server: ssh exited with exit code 12 PASS 77h (7s) == sanityn test 77i: Change rank of TBF rule ============= 19:10:52 (1746141052) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (15s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 19:11:07 (1746141067) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.345683 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900923 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902909 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316063 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241543 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293503 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552106 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0722253 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.238028 s, 4.4 MB/s 1048576 bytes (1.0 MB) copied, 0.215175 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.249948 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.222579 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.257551 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.230018 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.248418 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.279475 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.291573 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278112 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259638 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297066 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501851 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345713 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375912 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379574 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378825 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403614 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446994 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484714 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051857 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502181 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532178 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0553154 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0642315 s, 16.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.91621 s, 13.2 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.5198 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (57s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 19:12:04 (1746141124) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.26343 s, 13.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152493 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039766 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0916782 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643615 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0800653 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0796861 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0759066 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106118 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0681323 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0974029 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105006 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146299 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166138 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159042 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164886 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185197 s, 5.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0312156 s, 33.6 MB/s , 0.0393781 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03234 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335121 s, 31.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470518 s, 22.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0383586 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392128 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416713 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449283 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562576 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514128 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567562 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057782 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.059171 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589834 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.063448 s, 16.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8518 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9027 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.504514 s, 33.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037339 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361913 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494805 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041504 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0750668 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0815746 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0877405 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111683 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10432 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106069 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117636 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153351 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156712 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165035 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165313 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197914 s, 5.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268646 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372224 s, 28.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301521 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283223 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342562 s, 30.6 MB/s , 0.0304471 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339588 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384235 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382474 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372438 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408503 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406689 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464253 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492078 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490173 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0641701 s, 16.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9678 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8971 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (140s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 19:14:24 (1746141264) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.27323 s, 13.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.06462 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0775688 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0939205 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0899478 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0979917 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103631 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122556 s, 8.6 MB/s 1048576 bytes (1.0 MB) copied, 0.128669 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124382 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13711 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13964 s, 7.5 MB/s , 0.127363 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14305 s, 7.3 MB/s 1048576 bytes (1.0 MB) copied, 0.146002 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172798 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.201164 s, 5.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469745 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494444 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467517 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518629 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508824 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539292 s, 19.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05476 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0545426 s, 19.2 MB/s , 0.0557921 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0577441 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567505 s, 18.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0623416 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589522 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0631417 s, 16.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0676962 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0689944 s, 15.2 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8233 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9147 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg121-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.540868 s, 31.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371389 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431516 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0563499 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0710732 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538304 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102564 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121991 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144474 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142439 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15047 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174989 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176282 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164183 s, 6.4 MB/s , 0.156411 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176044 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177166 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319493 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291224 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267507 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271422 s, 38.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324355 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348044 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371258 s, 28.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0441346 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423312 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479897 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485831 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514686 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0523126 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529916 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570371 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659513 s, 15.9 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8336 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.919 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (137s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 19:16:41 (1746141401) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.41893 s, 40.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020641 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249305 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0653054 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0655041 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0700554 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.089373 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0918034 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0836863 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0936134 s, 11.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.100814 s, 10.4 MB/s , 0.104619 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112294 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134648 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131617 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149857 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161481 s, 6.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288496 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318548 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00268429 s, 391 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514507 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519182 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369081 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326923 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0364079 s, 28.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00256569 s, 409 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569384 s, 18.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051201 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930664 s, 113 MB/s , 0.0438156 s, 23.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562687 s, 18.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0133579 s, 78.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0605865 s, 17.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.14865 s, 14.7 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81007 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.21@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.21@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.340928 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161123 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017175 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0580192 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603911 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403188 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0903055 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100947 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114921 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126384 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129634 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127909 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140316 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190689 s, 5.5 MB/s 1048576 bytes (1.0 MB) copied, 0.179531 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192654 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.203314 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00335493 s, 313 MB/s 1048576 bytes (1.0 MB) copied, 0.00797918 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168086 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00331163 s, 317 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0047271 s, 222 MB/s , 0.00492287 s, 213 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00220259 s, 476 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00271541 s, 386 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00348853 s, 301 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00486456 s, 216 MB/s , 0.00673255 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565474 s, 18.5 MB/s , 0.0102016 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257407 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227719 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00196855 s, 533 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.37444 s, 16.4 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81699 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.21@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.372795 s, 45.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216569 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128996 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0601038 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0521961 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.092419 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0789098 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0858649 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181931 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141943 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153206 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11484 s, 9.1 MB/s 1048576 bytes (1.0 MB) copied, 0.194176 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130556 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158529 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167578 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.266819 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539094 s, 19.5 MB/s , 0.0381998 s, 27.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0412072 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438309 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448138 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0578385 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449512 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455502 s, 23.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0616718 s, 17.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0753726 s, 13.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0625564 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633193 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0651718 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643614 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0681112 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0718897 s, 14.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 8.21829 s, 12.8 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.01124 s, 20.9 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.289495 s, 58.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298018 s, 35.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0308773 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568758 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0557271 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140342 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112865 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174111 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.13976 s, 7.5 MB/s 1048576 bytes (1.0 MB) copied, 0.152676 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.195588 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186521 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171979 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176405 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182912 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161764 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18264 s, 5.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187899 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219216 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327267 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275684 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307692 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315079 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334186 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335324 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360868 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360982 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445502 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043045 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504601 s, 20.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477984 s, 21.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0481808 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568589 s, 18.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 10.2311 s, 10.2 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.87028 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.49882 s, 16.1 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.80549 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.316585 s, 53.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294497 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314354 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0688333 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0802431 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0857829 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0833117 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0806045 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0895999 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.094002 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.089957 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0969559 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100908 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113542 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115475 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113342 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130987 s, 8.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370985 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.050105 s, 20.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0501447 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348418 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03775 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404941 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413711 s, 25.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0446416 s, 23.5 MB/s , 0.0501832 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462755 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500197 s, 21.0 MB/s , 0.0440407 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524009 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442681 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404208 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603692 s, 17.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7316 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.5126 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.292221 s, 57.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0195383 s, 53.7 MB/s , 0.0208327 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0566667 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0776567 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0924042 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0988958 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.086522 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100438 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104972 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0910085 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0977481 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0901902 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123255 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123324 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118021 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142735 s, 7.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377716 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378562 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05451 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0573852 s, 18.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0119607 s, 87.7 MB/s , 0.0201022 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0138376 s, 75.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0141698 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300767 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163006 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187943 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198543 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264065 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272083 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305733 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396649 s, 26.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.93801 s, 17.7 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.845 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.298987 s, 56.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199203 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300436 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596596 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0635077 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0830723 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0898409 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0945222 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.089285 s, 11.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112654 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108333 s, 9.7 MB/s 1048576 bytes (1.0 MB) copied, 0.111695 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113778 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197263 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.195673 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.274796 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.270085 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356582 s, 29.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0356193 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371978 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292754 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313774 s, 33.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511612 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391924 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568881 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427098 s, 24.6 MB/s 1048576 bytes (1.0 MB) copied, 0.048716 s, 21.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0500103 s, 21.0 MB/s , 0.0502818 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511022 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0533622 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052913 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0584195 s, 17.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 10.2159 s, 10.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.83969 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 10.1751 s, 10.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.83042 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (381s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 19:23:02 (1746141782) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (7s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 19:23:09 (1746141789) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg121-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.79842 s, 276 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg121-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0036 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (53s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 19:24:02 (1746141842) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.218528 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359574 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389633 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0912696 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0993902 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0962122 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108877 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102308 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116507 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106972 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110257 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121052 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111808 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117929 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132955 s, 7.9 MB/s 1048576 bytes (1.0 MB) copied, 0.130321 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137484 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124304 s, 84.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0478166 s, 21.9 MB/s , 0.041182 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0025292 s, 415 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00279979 s, 375 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340355 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040058 s, 26.2 MB/s 1+0 records in 1+0 records out , 0.0501236 s, 20.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0281766 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222621 s, 47.1 MB/s , 0.0245572 s, 42.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0254304 s, 41.2 MB/s , 0.0100441 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00355304 s, 295 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117138 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0531285 s, 19.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg121-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.80738 s, 18.1 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.99188 s, 21.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.357164 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240415 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305397 s, 34.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0207231 s, 50.6 MB/s , 0.0378172 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165113 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168098 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170702 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179249 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177719 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188299 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160255 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177631 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.230189 s, 4.6 MB/s , 0.232983 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.240585 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.263643 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383214 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00241964 s, 433 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826637 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00429769 s, 244 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00286482 s, 366 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105935 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0831426 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0668887 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0606026 s, 17.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0411589 s, 25.5 MB/s , 0.0478674 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361251 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.074787 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295078 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127263 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877556 s, 119 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg121-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg121-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.37317 s, 14.2 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.00391 s, 21.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (73s) == sanityn test 77o: Changing rank should not panic ====== 19:25:15 (1746141915) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (7s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 19:25:22 (1746141922) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (55s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 19:26:17 (1746141977) mds.MDS.mdt.nrs_policies=tbf oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg121-client: oleg121-server: ssh exited with exit code 36 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg121-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg121-client: oleg121-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (20s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 19:26:37 (1746141997) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 21438 kill md_thread xargs PASS 77r (41s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 19:27:18 (1746142038) oleg121-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg121-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg121-client: oleg121-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (5s) == sanityn test 79: xattr: intent error ================== 19:27:23 (1746142043) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505743 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:27:24.000000000 -0400 Modify: 2025-05-01 19:27:24.000000000 -0400 Change: 2025-05-01 19:27:24.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (15s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 19:27:38 (1746142058) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 19:27:40 (1746142060) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 19:27:42 (1746142062) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 19:27:44 (1746142064) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 19:27:47 (1746142067) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 19:27:49 (1746142069) - open/close 1016 (time 1746142082.99 total 10.01 last 101.52) - open/close 1935 (time 1746142093.01 total 20.02 last 91.78) total: 2500 open/close in 25.18 seconds: 99.30 ops/second snapshot_time 1746142152.275411023 secs.nsecs start_time 1746142071.374095059 secs.nsecs elapsed_time 80.901315964 secs.nsecs open 2505 samples [usecs] 116 10599 2435016 4197581832 close 2504 samples [usecs] 76 4428 863331 618960791 mknod 2501 samples [usecs] 251 10067 1990973 2735418889 unlink 2500 samples [usecs] 387 63167 4505011 31374962405 mkdir 2 samples [usecs] 1364 2035 3399 6001721 rmdir 2 samples [usecs] 8503 11204 19707 197830625 rename 2500 samples [usecs] 321 24831 10204416 75011082420 getattr 3762 samples [usecs] 5 3268 152962 76319606 parallel_rename_file 2500 samples [usecs] 321 24831 10204416 75011082420 crossdir_rename 2500 samples [usecs] 321 24831 10204416 75011082420 PASS 81d (87s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 19:29:16 (1746142156) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7063 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7063 PASS 82 (5s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 19:29:21 (1746142161) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (3s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 19:29:24 (1746142164) fail_loc=0x8000060b PASS 84 (11s) == sanityn test 85: Lustre API root cache race =========== 19:29:35 (1746142175) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1746142176 Finishing test test1 at 1746142179 PASS 85 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 19:29:42 (1746142182) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (3s) == sanityn test 91: chmod and unlink striped directory === 19:29:45 (1746142185) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) == sanityn test 92: create remote directory under orphan directory ========================================================== 19:29:47 (1746142187) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (3s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 19:29:50 (1746142190) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5122 0x1402 0x240000400 1 5107 0x13f3 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5108 0x13f4 0x280000400 0 5123 0x1403 0x240000400 rc1=0 and rc2=0 PASS 93 (12s) == sanityn test 94: signal vs CP callback race =========== 19:30:02 (1746142202) osc.lustre-OST0000-osc-ffff8800aa034800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800af65b000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa034800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800af65b000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.155542 s, 52.7 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00351 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5764: 4075 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0850226 s, 48.2 kB/s osc.lustre-OST0000-osc-ffff8800aa034800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800af65b000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa034800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800af65b000.idle_timeout=debug PASS 94 (13s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 19:30:15 (1746142215) llite.lustre-ffff8800aa034800.fast_read=0 llite.lustre-ffff8800af65b000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0491735 s, 42.6 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800aa034800.fast_read=1 llite.lustre-ffff8800af65b000.fast_read=1 PASS 95a (21s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 19:30:36 (1746142236) llite.lustre-ffff8800aa034800.fast_read=0 llite.lustre-ffff8800af65b000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.0116615 s, 1.1 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa034800.fast_read=1 llite.lustre-ffff8800af65b000.fast_read=1 PASS 95b (11s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 19:30:47 (1746142247) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 19:30:49 (1746142249) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.009292 s, 441 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (4s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 19:30:53 (1746142253) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0628121 s, 33.4 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (4s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 19:30:58 (1746142258) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0739721 s, 28.4 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (5s) == sanityn test 100e: DoM: read on open and file size ==== 19:31:02 (1746142262) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00468784 s, 2.6 MB/s PASS 100e (5s) == sanityn test 101a: Discard DoM data on unlink ========= 19:31:07 (1746142267) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00124438 s, 3.3 MB/s PASS 101a (5s) == sanityn test 101b: Discard DoM data on rename ========= 19:31:12 (1746142272) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0102599 s, 399 kB/s PASS 101b (6s) == sanityn test 101c: Discard DoM data on close-unlink === 19:31:18 (1746142278) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00174212 s, 2.4 MB/s PASS 101c (6s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 19:31:25 (1746142285) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1746142287 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1746142291 Finishing test test23 at 1746142291 PASS 103 (12s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 19:31:37 (1746142297) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 19:31:39 (1746142299) fail_loc=0x416 PASS 105 (29s) == sanityn test 106a: Verify the btime via statx() ======= 19:32:09 (1746142329) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (3s) == sanityn test 106b: Glimpse RPCs test for statx ======== 19:32:11 (1746142331) SKIP: sanityn test_106b statx() only test SKIP 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 19:32:13 (1746142333) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 19:32:14 (1746142334) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.140349 s, 74.7 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5130 0x140a 0x240000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 First grouplock blocks second one PASS 107a (6s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 19:32:20 (1746142340) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.155112 s, 67.6 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5114 0x13fa 0x280000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7063 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (11s) == sanityn test 108a: lseek: parallel updates ============ 19:32:31 (1746142351) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00238616 s, 4.2 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.40964 s, 1.9 MB/s PASS 108a (8s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 19:32:39 (1746142359) 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 3 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 4 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 6 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 8 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 9 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/pcc_cpufreq/refcnt': 'No such device' Iteration 12 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 13 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 16 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 32 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 34 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 41 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 43 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 44 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 45 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 46 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 192.168.201.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.121@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg121-server' oleg121-server: oleg121-server.virtnet: executing load_modules_local oleg121-server: Loading modules from /home/green/git/lustre-release/lustre oleg121-server: detected 4 online CPUs by sysfs oleg121-server: MODOPTS_LIBCFS= oleg121-server: Force libcfs to create 2 CPU partitions oleg121-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre Starting client: oleg121-client.virtnet: -o user_xattr,flock 192.168.201.121@tcp:/lustre /mnt/lustre2 PASS 109 (763s) == sanityn test 110: do not grant another lock on resend ========================================================== 19:45:22 (1746143122) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 19:45:23.000000000 -0400 Modify: 2025-05-01 19:45:23.000000000 -0400 Change: 2025-05-01 19:45:23.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg121-client.virtnet oleg121-client.virtnet PASS 110 (67s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 19:46:29 (1746143189) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (3s) == sanityn test 112: update max-inherit in default LMV === 19:46:32 (1746143192) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (2s) == sanityn test 113: check servers of specified fs ======= 19:46:34 (1746143194) lustre-OST0000-osc-ffff8800a9214000 active. lustre-OST0001-osc-ffff8800a9214000 active. lustre-OST0000-osc-ffff8800a9214000 active. lustre-OST0001-osc-ffff8800a9214000 active. MGC192.168.201.121@tcp active. PASS 113 (4s) == sanityn test 114: implicit default LMV inherit ======== 19:46:38 (1746143198) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (3s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 19:46:41 (1746143201) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (2s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 19:46:43 (1746143203) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (2s) == sanityn test 200: service remains healthy while able to process request ========================================================== 19:46:45 (1746143205) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=20 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1533 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.2163 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1628 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.12 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1535 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1281 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1482 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1821 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1788 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2009 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1936 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2058 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.223 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1861 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1137 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2326 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1279 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2977 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2307 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.281 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.3488 s, 0.1 kB/s , 60.2677 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 60.1921 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 60.3421 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2918 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.283 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4324 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2537 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3029 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3301 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2842 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4855 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3495 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3729 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3989 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3614 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.442 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.415 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.423 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.575 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg121-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9214000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9214000.ost_server_uuid in FULL state after 0 sec PASS 200 (113s) cleanup: ====================================================== == sanityn test complete, duration 7595 sec ============== 19:48:40 (1746143320) sanityn: FAIL: test_16j dio copy 1 w/bsize 1049600 failed: 1 === sanityn: start cleanup 19:48:41 (1746143321) === Stopping clients: oleg121-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg121-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 19:51:35 (1746143495) ===