-----============= acceptance-small: sanityn ============----- Sun Jun 7 00:51:25 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg211-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a 102 skipping tests SLOW=no: 33a === sanityn: start setup 00:51:34 (1780807894) === Starting client oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 Mount client oleg211-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 Started clients oleg211-client.virtnet: 192.168.202.111@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg211-client.virtnet: executing check_config_client /mnt/lustre oleg211-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg211-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8ab686465000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8ab687ae7800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ab686465000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ab687ae7800.idle_timeout=debug disable quota as required oleg211-server: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 00:51:49 (1780807909) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154685 s, 67.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7564] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 00:51:50 (1780807910) PASS 0 (3s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 00:51:53 (1780807913) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 00:51:56 (1780807916) -rw-r--r-- 1 root root 0 Jun 7 00:51 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 00:51:58 (1780807918) -rw-r--r-- 1 root root 0 Jun 7 00:51 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 00:52:01 (1780807921) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 00:52:04 (1780807924) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 00:52:07 (1780807927) total 1 -rwxrwxrwx 1 root root 0 Jun 7 00:51 f2a -rwxrwxrwx 1 root root 0 Jun 7 00:51 f2b total 1 -rwxrwxrwx 1 root root 0 Jun 7 00:51 f2a -rwxrwxrwx 1 root root 0 Jun 7 00:51 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00603633 s, 84.8 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 00:52:10 (1780807930) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 00:52:11 (1780807931) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.200474 s, 10.5 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 00:52:15 (1780807935) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 00:52:18 (1780807938) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 00:52:23 (1780807943) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 00:52:25 (1780807945) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 00:52:29 (1780807949) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (4s) == sanityn test 8: remove of open special file on other node ==================================================================== 00:52:33 (1780807953) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 00:52:37 (1780807957) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 00:52:40 (1780807960) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181963 s, 57.6 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0153253 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 00:52:43 (1780807963) 1+0 records in 1+0 records out 1 byte copied, 0.0073787 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0101925 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00770607 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0142791 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00783831 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00936738 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0108418 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00821232 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00631967 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00519915 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0047522 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00691701 s, 0.1 kB/s PASS 10a (4s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 00:52:47 (1780807967) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00416525 s, 738 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0190489 s, 215 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000231122 s, 13.3 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 00:52:50 (1780807970) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 00:52:54 (1780807974) warning: '-runtime' deprecated, use '-t runtime' instead - link 672 (time 1780807985.86 total 10.01 last 67.15) - link 1277 (time 1780807995.86 total 20.01 last 60.49) - link 2035 (time 1780808005.87 total 30.02 last 75.73) - link 2836 (time 1780808015.88 total 40.03 last 80.02) - link 3635 (time 1780808025.89 total 50.04 last 79.84) - link 4547 (time 1780808035.89 total 60.04 last 91.15) - link 5617 (time 1780808045.90 total 70.05 last 106.93) - link 6586 (time 1780808055.91 total 80.06 last 96.80) - link 7326 (time 1780808065.92 total 90.07 last 73.93) total: 8025 link in 100.01 seconds: 80.24 ops/second using seed 11187451 running for 100 seconds - stat 9518 (time 1780807987 ; total 11 ; last 11) - stat 10000 (time 1780807987 ; total 11 ; last 0) - stat 18514 (time 1780807998 ; total 22 ; last 11) - stat 20000 (time 1780807999 ; total 23 ; last 1) - stat 30000 (time 1780808008 ; total 32 ; last 9) - stat 39866 (time 1780808019 ; total 43 ; last 11) - stat 40000 (time 1780808019 ; total 43 ; last 0) - stat 49836 (time 1780808030 ; total 54 ; last 11) - stat 50000 (time 1780808030 ; total 54 ; last 0) - stat 55039 (time 1780808041 ; total 65 ; last 11) - stat 60000 (time 1780808051 ; total 75 ; last 10) - stat 64497 (time 1780808062 ; total 86 ; last 11) - stat 68796 (time 1780808073 ; total 97 ; last 11) total: 69980 stats in 100 seconds: 699.799988 stats/second - unlinked 0 (time 1780808026 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile8025) error: No such file or directory total: 8024 unlinks in 101 seconds: 79.445541 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21332) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21334) - No such process PASS 12 (156s) == sanityn test 13: test directory page revocation ======= 00:55:30 (1780808130) f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 00:55:34 (1780808134) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 22520 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 00:55:38 (1780808138) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 23098 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 00:55:41 (1780808141) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 23677 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 00:55:44 (1780808144) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 24258 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 00:55:47 (1780808147) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 24840 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 00:55:51 (1780808151) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky sleep 5 for ZFS MDS Waiting for MDT destroys to complete fallocate on zfs doesn't consume space STRIPECOUNT=2 ORIGFREE=7527424 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7527424KB free > 800000KB, increase MAXFREE (or reduce fs size) sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (34s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 00:56:25 (1780808185) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.16642 s, 50.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.113412 s, 74.0 MB/s Chance of close/open is 1 in 50 Seed set to 8186 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1780808187.264436 mapwrite 0x747497 thru 0x749e08 (0x2972 bytes) 000003[0] 1780808187.324388 write 0x954be5 thru 0x961725 (0xcb41 bytes) 000004[1] 1780808187.338441 read 0x6bec3a thru 0x6cbbfb (0xcfc2 bytes) 000007[1] 1780808187.368586 mapread 0x14e90f thru 0x15cf1e (0xe610 bytes) 000008[1] 1780808187.401067 write 0x8cd0e2 thru 0x8d5126 (0x8045 bytes) 000009[1] 1780808187.411038 read 0x8fc05b thru 0x8fd950 (0x18f6 bytes) truncating to largest ever: 0x3ae5b2 000010[0] 1780808187.431439 trunc from 0x961726 to 0x3ae5b1 (0x5b3174 bytes) 000011[0] 1780808187.474628 mapwrite 0x721060 thru 0x7289dc (0x797d bytes) 000014[0] 1780808187.554570 read 0x248c2d thru 0x2505e3 (0x79b7 bytes) 000017[0] 1780808187.570525 mapwrite 0x075e05 thru 0x084882 (0xea7e bytes) 000018[1] 1780808187.646870 read 0x72134d thru 0x7289dc (0x7690 bytes) 000021[0] 1780808187.698456 mapread 0x6d6b02 thru 0x6deb26 (0x8025 bytes) 000022[1] 1780808187.714003 read 0x0b146b thru 0x0bd613 (0xc1a9 bytes) 000024[0] 1780808187.737773 mapwrite 0x21c37b thru 0x225e6b (0x9af1 bytes) 000025[1] 1780808187.793953 write 0x3dda9b thru 0x3e8534 (0xaa9a bytes) 000027[1] 1780808187.825830 write 0x226097 thru 0x228306 (0x2270 bytes) 000028[1] 1780808187.849244 write 0x23aa08 thru 0x2471d1 (0xc7ca bytes) 000030[0] 1780808187.866400 write 0x5354ac thru 0x53ea2f (0x9584 bytes) 000033[1] 1780808187.913397 mapread 0x60c501 thru 0x610a20 (0x4520 bytes) 000035[0] 1780808187.945236 write 0x049992 thru 0x057668 (0xdcd7 bytes) 000036[0] 1780808187.970834 read 0x6304da thru 0x6350ea (0x4c11 bytes) 000040[1] 1780808187.982540 mapread 0x432105 thru 0x43a793 (0x868f bytes) 000041[0] 1780808187.997664 write 0x40b034 thru 0x419d57 (0xed24 bytes) 000042[0] 1780808188.011887 mapwrite 0x2a44a2 thru 0x2a6922 (0x2481 bytes) 000044[0] 1780808188.074894 read 0x067e77 thru 0x0710ec (0x9276 bytes) 000045[1] 1780808188.082014 read 0x5d5a08 thru 0x5d6628 (0xc21 bytes) truncating to largest ever: 0x610931 000047[1] 1780808188.097977 trunc from 0x7289dd to 0x610930 (0x1180ac bytes) 000050[1] 1780808188.130582 read 0x4c3167 thru 0x4cbb0a (0x89a4 bytes) 000051[0] 1780808188.141887 read 0x044e72 thru 0x047fba (0x3149 bytes) 000052[1] 1780808188.150923 write 0x354946 thru 0x361d67 (0xd422 bytes) 000055[1] 1780808188.178842 write 0x7312e9 thru 0x739c23 (0x893b bytes) 000057[0] 1780808188.190364 write 0x3900bc thru 0x391c18 (0x1b5d bytes) 000058[1] 1780808188.214642 read 0x014fdc thru 0x01c6cb (0x76f0 bytes) 000059[0] 1780808188.245894 mapread 0x17bc27 thru 0x17d194 (0x156e bytes) 000061[0] 1780808188.288855 trunc from 0x739c24 to 0x2a6520 (0x493703 bytes) 000062[0] 1780808188.321001 write 0x3091e2 thru 0x30a0b8 (0xed7 bytes) 000064[0] 1780808188.341073 trunc from 0x30a0b9 to 0x29a24d (0x6fe6b bytes) 000067[0] 1780808188.378357 mapwrite 0x627218 thru 0x628079 (0xe62 bytes) 000068[0] 1780808188.436042 mapread 0x23cf77 thru 0x2416e8 (0x4772 bytes) truncating to largest ever: 0x90e887 000069[0] 1780808188.454061 trunc from 0x62807a to 0x90e886 (0x2e680d bytes) 000071[1] 1780808188.484295 write 0x4509f8 thru 0x45d89a (0xcea3 bytes) 000072[0] 1780808188.499122 mapread 0x0e3d23 thru 0x0ec886 (0x8b64 bytes) 000076[0] 1780808188.544903 write 0x8f1859 thru 0x8f9003 (0x77ab bytes) 000081[1] 1780808188.596486 read 0x6860e2 thru 0x691561 (0xb480 bytes) 000083[1] 1780808188.624840 trunc from 0x90e887 to 0x56073f (0x3ae147 bytes) 000084[1] 1780808188.642424 mapread 0x354aaf thru 0x360718 (0xbc6a bytes) 000085[1] 1780808188.667463 mapwrite 0x321e77 thru 0x32a43f (0x85c9 bytes) 000086[0] 1780808188.698420 mapwrite 0x0cd27b thru 0x0cdb4c (0x8d2 bytes) 000087[1] 1780808188.748141 mapread 0x1eaa18 thru 0x1ef3fd (0x49e6 bytes) 000089[1] 1780808188.777554 write 0x9418dd thru 0x9439a9 (0x20cd bytes) 000090[1] 1780808188.783907 trunc from 0x9439aa to 0x1fcf18 (0x746a91 bytes) 000091[0] 1780808188.800901 mapread 0x144bc6 thru 0x1465ae (0x19e9 bytes) 000093[1] 1780808188.822245 write 0x25b39d thru 0x26010f (0x4d73 bytes) 000095[1] 1780808188.832036 read 0x199fed thru 0x19c206 (0x221a bytes) 000096[1] 1780808188.848545 read 0x242706 thru 0x249b1c (0x7417 bytes) 000101[1] 1780808188.861039 read 0x19d8b9 thru 0x1a346d (0x5bb5 bytes) 000102[0] 1780808188.861593 read 0x16aa5e thru 0x1725d0 (0x7b73 bytes) 000103[0] 1780808188.897752 read 0x193a89 thru 0x195080 (0x15f8 bytes) 000104[0] 1780808188.902533 write 0x490e05 thru 0x498cf2 (0x7eee bytes) 000105[0] 1780808188.915129 read 0x13e792 thru 0x146c87 (0x84f6 bytes) 000106[1] 1780808188.915988 mapwrite 0x83c9f3 thru 0x8433e8 (0x69f6 bytes) 000107[0] 1780808189.022765 trunc from 0x8433e9 to 0x59d948 (0x2a5aa0 bytes) 000108[0] 1780808189.043385 trunc from 0x59d949 to 0x56cf8e (0x309ba bytes) 000110[1] 1780808189.049760 mapwrite 0x42da7c thru 0x439d5d (0xc2e2 bytes) 000111[0] 1780808189.134875 write 0x3ac886 thru 0x3ba70c (0xde87 bytes) 000114[1] 1780808189.170587 mapread 0x0e0340 thru 0x0e4cd9 (0x499a bytes) 000115[1] 1780808189.204500 mapwrite 0x71ae80 thru 0x724a39 (0x9bba bytes) 000117[0] 1780808189.312929 trunc from 0x724a3a to 0x8acb10 (0x1880d7 bytes) 000118[0] 1780808189.347721 trunc from 0x8acb11 to 0x045cf0 (0x866e20 bytes) 000119[0] 1780808189.372236 mapwrite 0x1aba9e thru 0x1bb117 (0xf67a bytes) 000120[1] 1780808189.454228 trunc from 0x1bb118 to 0x4d445c (0x319345 bytes) 000121[0] 1780808189.482533 mapread 0x2ec48e thru 0x2ef3e2 (0x2f55 bytes) 000124[0] 1780808189.514683 trunc from 0x4d445d to 0x6f8edd (0x224a81 bytes) 000128[0] 1780808189.579709 mapread 0x559bd6 thru 0x562d6c (0x9197 bytes) 000129[1] 1780808189.607975 read 0x43631d thru 0x445d0c (0xf9f0 bytes) 000130[1] 1780808189.637669 mapread 0x3087c9 thru 0x30b702 (0x2f3a bytes) 000131[1] 1780808189.672994 mapread 0x571283 thru 0x572cce (0x1a4c bytes) 000132[1] 1780808189.687587 mapwrite 0x10109c thru 0x10ac2b (0x9b90 bytes) 000133[1] 1780808189.738038 write 0x9e9855 thru 0x9f66b1 (0xce5d bytes) 000136[0] 1780808189.774110 read 0x27da5c thru 0x284fe2 (0x7587 bytes) 000137[0] 1780808189.789547 mapread 0x1f1461 thru 0x1f1c9a (0x83a bytes) 000142[0] 1780808189.794561 write 0x7d805c thru 0x7de7de (0x6783 bytes) 000145[0] 1780808189.817176 read 0x745221 thru 0x74e96c (0x974c bytes) 000147[0] 1780808189.827914 mapwrite 0x43e2c4 thru 0x447c84 (0x99c1 bytes) 000151[1] 1780808189.865393 mapwrite 0x7600a6 thru 0x7650ce (0x5029 bytes) 000153[0] 1780808189.935918 mapread 0x9386ba thru 0x93e7e3 (0x612a bytes) 000154[0] 1780808189.954445 mapread 0x220125 thru 0x22aada (0xa9b6 bytes) 000155[0] 1780808189.970375 trunc from 0x9f66b2 to 0x82156c (0x1d5145 bytes) 000156[1] 1780808190.003861 mapwrite 0x5acd3a thru 0x5ba2cc (0xd593 bytes) 000158[1] 1780808190.101590 write 0x2fc1f4 thru 0x30a5cf (0xe3dc bytes) 000160[1] 1780808190.130600 mapread 0x56d5e8 thru 0x56f54a (0x1f63 bytes) 000163[1] 1780808190.140530 mapread 0x4b6acd thru 0x4bbc37 (0x516b bytes) 000164[0] 1780808190.149421 read 0x654bbe thru 0x6574ef (0x2932 bytes) 000165[0] 1780808190.161151 mapwrite 0x3df74a thru 0x3ee483 (0xed3a bytes) 000170[1] 1780808190.212356 read 0x632cb6 thru 0x64277b (0xfac6 bytes) 000171[0] 1780808190.216795 mapwrite 0x0f75be thru 0x0fb6f1 (0x4134 bytes) 000174[0] 1780808190.252507 write 0x28da60 thru 0x29d64e (0xfbef bytes) 000175[0] 1780808190.273059 trunc from 0x82156d to 0x61ef2c (0x202640 bytes) 000176[0] 1780808190.304989 write 0x4235ff thru 0x429d99 (0x679b bytes) 000177[0] 1780808190.322564 read 0x572d00 thru 0x578feb (0x62ec bytes) 000180[0] 1780808190.347456 trunc from 0x61ef2d to 0x64e074 (0x2f148 bytes) 000184[0] 1780808190.377041 write 0x55f02e thru 0x5685a5 (0x9578 bytes) 000190[0] 1780808190.400512 mapwrite 0x50e9ba thru 0x51c0cc (0xd713 bytes) 000191[1] 1780808190.437122 write 0x58c9bc thru 0x595e79 (0x94be bytes) 000193[0] 1780808190.467800 trunc from 0x64e075 to 0x7c144f (0x1733db bytes) 000196[0] 1780808190.512912 mapwrite 0x65a151 thru 0x665481 (0xb331 bytes) 000197[0] 1780808190.565083 write 0x6c1bde thru 0x6c96c1 (0x7ae4 bytes) 000198[1] 1780808190.587046 write 0x4bd3bf thru 0x4c4d69 (0x79ab bytes) truncating to largest ever: 0x960de3 000200[1] 1780808190.618712 trunc from 0x7c1450 to 0x960de2 (0x19f993 bytes) 000205[1] 1780808190.666456 trunc from 0x960de3 to 0x4ca08a (0x496d58 bytes) 000207[1] 1780808190.694640 mapread 0x007c6b thru 0x00ed35 (0x70cb bytes) 000212[0] 1780808190.763728 mapwrite 0x5f254a thru 0x5fcb09 (0xa5c0 bytes) 000213[0] 1780808190.834348 write 0x7e75bf thru 0x7ea0d5 (0x2b17 bytes) 000219[1] 1780808190.873334 mapwrite 0x533707 thru 0x539025 (0x591f bytes) 000221[1] 1780808190.964596 mapread 0x489964 thru 0x48f88c (0x5f29 bytes) 000222[1] 1780808190.995681 trunc from 0x7ea0d6 to 0x8863ca (0x9c2f5 bytes) 000223[0] 1780808191.016757 trunc from 0x8863cb to 0x87fa59 (0x6971 bytes) 000228[1] 1780808191.108404 mapwrite 0x0f99c4 thru 0x10048f (0x6acc bytes) 000232[1] 1780808191.210212 mapread 0x0b6cea thru 0x0c2fd8 (0xc2ef bytes) 000233[1] 1780808191.236783 mapread 0x7aa64e thru 0x7aec3e (0x45f1 bytes) 000236[1] 1780808191.262232 write 0x0ad8bc thru 0x0b339e (0x5ae3 bytes) 000238[1] 1780808191.267480 mapread 0x5dd5fe thru 0x5dde54 (0x857 bytes) 000239[0] 1780808191.274634 write 0x281fab thru 0x28b4de (0x9534 bytes) 000240[1] 1780808191.321011 trunc from 0x87fa5a to 0x0bdac8 (0x7c1f91 bytes) 000242[0] 1780808191.372162 mapwrite 0x5cc71c thru 0x5d2152 (0x5a37 bytes) 000243[1] 1780808191.477131 trunc from 0x5d2153 to 0x68b617 (0xb94c5 bytes) 000244[1] 1780808191.516787 trunc from 0x68b618 to 0x07fc01 (0x60ba16 bytes) 000246[0] 1780808191.552080 trunc from 0x07fc02 to 0x7a11e4 (0x7215e3 bytes) 000250[1] 1780808191.593856 write 0x89b2ed thru 0x8a98aa (0xe5be bytes) 000251[0] 1780808191.610647 write 0x01d0f6 thru 0x01ee6d (0x1d78 bytes) 000253[0] 1780808191.656613 write 0x1c2161 thru 0x1d1b3b (0xf9db bytes) 000255[1] 1780808191.690122 read 0x0692e2 thru 0x072d1f (0x9a3e bytes) 000258[1] 1780808191.716623 write 0x183840 thru 0x1920c2 (0xe883 bytes) 000260[1] 1780808191.747169 mapread 0x00cb01 thru 0x01af08 (0xe408 bytes) 000261[1] 1780808191.783526 read 0x2ee2ab thru 0x2f7e93 (0x9be9 bytes) 000265[0] 1780808191.841516 trunc from 0x8a98ab to 0x4d28e3 (0x3d6fc7 bytes) 000266[0] 1780808191.874823 trunc from 0x4d28e4 to 0x4d19e4 (0xeff bytes) 000271[1] 1780808191.940166 read 0x1a09a2 thru 0x1a3416 (0x2a75 bytes) 000272[0] 1780808191.959430 read 0x2f3fcd thru 0x2f6019 (0x204d bytes) 000274[1] 1780808191.973323 mapwrite 0x970de3 thru 0x97ef61 (0xe17f bytes) 000275[0] 1780808192.052685 mapread 0x3f4565 thru 0x3fb09a (0x6b36 bytes) 000277[0] 1780808192.093907 read 0x0a9607 thru 0x0b1889 (0x8283 bytes) 000278[1] 1780808192.108833 mapwrite 0x8cc982 thru 0x8d4504 (0x7b83 bytes) 000279[0] 1780808192.144683 trunc from 0x97ef62 to 0x041ef0 (0x93d071 bytes) 000282[0] 1780808192.168998 read 0x03f546 thru 0x041ef0 (0x29ab bytes) 000283[1] 1780808192.173392 write 0x106ffa thru 0x116907 (0xf90e bytes) 000284[0] 1780808192.186406 write 0x52ec2d thru 0x53515c (0x6530 bytes) 000285[0] 1780808192.195166 mapwrite 0x776de2 thru 0x77a234 (0x3453 bytes) 000288[1] 1780808192.312321 mapread 0x63192c thru 0x63a2ed (0x89c2 bytes) 000290[0] 1780808192.361573 mapread 0x62ce80 thru 0x63c40b (0xf58c bytes) 000292[0] 1780808192.416793 write 0x99f934 thru 0x9a791b (0x7fe8 bytes) 000294[1] 1780808192.430729 write 0x7e6ee1 thru 0x7f404b (0xd16b bytes) 000296[1] 1780808192.458717 read 0x5921b6 thru 0x594740 (0x258b bytes) 000297[0] 1780808192.474848 read 0x5032f8 thru 0x507498 (0x41a1 bytes) 000298[0] 1780808192.494609 trunc from 0x9a791c to 0x2e021f (0x6c76fc bytes) 000299[1] 1780808192.511243 read 0x0d5f56 thru 0x0d677c (0x827 bytes) 000301[0] 1780808192.545409 mapread 0x040233 thru 0x04c5fe (0xc3cc bytes) 000302[1] 1780808192.574917 trunc from 0x2e0220 to 0x88b580 (0x5ab361 bytes) 000310[0] 1780808192.632080 read 0x0e427e thru 0x0ebaa9 (0x782c bytes) 000311[1] 1780808192.662629 trunc from 0x88b581 to 0x5459f1 (0x345b8f bytes) 000313[1] 1780808192.690525 mapwrite 0x3a80fb thru 0x3b15a3 (0x94a9 bytes) 000315[0] 1780808192.754139 read 0x2607a9 thru 0x26fbb9 (0xf411 bytes) 000318[1] 1780808192.782489 read 0x061284 thru 0x066fb1 (0x5d2e bytes) 000319[1] 1780808192.798135 mapread 0x2a4017 thru 0x2a69b0 (0x299a bytes) 000320[1] 1780808192.834070 write 0x26e96c thru 0x26fe61 (0x14f6 bytes) 000323[0] 1780808192.909992 write 0x41aa0a thru 0x41edac (0x43a3 bytes) 000326[1] 1780808192.956636 read 0x17d36a thru 0x180577 (0x320e bytes) 000329[0] 1780808192.991992 write 0x383509 thru 0x38bedf (0x89d7 bytes) 000331[1] 1780808193.000227 trunc from 0x5459f2 to 0x7c6d9d (0x2813ac bytes) 000333[1] 1780808193.038635 write 0x34e68a thru 0x34fab8 (0x142f bytes) 000334[1] 1780808193.052189 mapread 0x76a997 thru 0x77a85d (0xfec7 bytes) 000335[1] 1780808193.096513 write 0x447e56 thru 0x44b437 (0x35e2 bytes) 000338[1] 1780808193.122739 read 0x32ef2b thru 0x33a8dd (0xb9b3 bytes) 000340[1] 1780808193.152809 read 0x0ef014 thru 0x0fbd3a (0xcd27 bytes) 000344[0] 1780808193.160190 read 0x4e6209 thru 0x4e64fb (0x2f3 bytes) 000345[0] 1780808193.179201 mapwrite 0x97a135 thru 0x987cd4 (0xdba0 bytes) 000347[0] 1780808193.294554 trunc from 0x987cd5 to 0x08547f (0x902855 bytes) 000349[1] 1780808193.319967 mapread 0x01b1ae thru 0x029cd2 (0xeb25 bytes) 000350[0] 1780808193.350954 mapread 0x07f5ed thru 0x08431e (0x4d32 bytes) 000353[1] 1780808193.373180 write 0x7aad2d thru 0x7b1347 (0x661b bytes) 000355[1] 1780808193.386354 mapwrite 0x993939 thru 0x993b16 (0x1de bytes) truncating to largest ever: 0x9a770f 000356[1] 1780808193.436835 trunc from 0x993b17 to 0x9a770e (0x13bf8 bytes) 000358[1] 1780808193.457780 mapwrite 0x212830 thru 0x2208f4 (0xe0c5 bytes) 000359[1] 1780808193.514015 trunc from 0x9a770f to 0x052989 (0x954d85 bytes) 000361[1] 1780808193.531916 mapread 0x027502 thru 0x03512f (0xdc2e bytes) 000362[0] 1780808193.553608 write 0x2cc741 thru 0x2d1737 (0x4ff7 bytes) 000363[0] 1780808193.570733 write 0x146c11 thru 0x14feca (0x92ba bytes) 000364[0] 1780808193.584059 mapwrite 0x23a8ac thru 0x249aab (0xf200 bytes) 000365[0] 1780808193.629450 mapwrite 0x54ac28 thru 0x550736 (0x5b0f bytes) 000367[1] 1780808193.699798 mapwrite 0x3fbc83 thru 0x40ac45 (0xefc3 bytes) 000368[1] 1780808193.775799 read 0x4de7e9 thru 0x4ea7fb (0xc013 bytes) 000369[0] 1780808193.779958 read 0x536327 thru 0x5364d4 (0x1ae bytes) 000370[0] 1780808193.787659 mapread 0x1ccfc1 thru 0x1d6ffa (0xa03a bytes) 000371[1] 1780808193.802040 read 0x1767f2 thru 0x17c48f (0x5c9e bytes) 000375[0] 1780808193.808900 write 0x8a22fb thru 0x8a7db0 (0x5ab6 bytes) 000376[0] 1780808193.816529 read 0x6b5d75 thru 0x6c01f5 (0xa481 bytes) 000378[0] 1780808193.860452 trunc from 0x8a7db1 to 0x20e38d (0x699a23 bytes) 000381[1] 1780808193.896115 mapwrite 0x6f9502 thru 0x706df4 (0xd8f3 bytes) 000383[0] 1780808193.961476 trunc from 0x706df5 to 0x8e9899 (0x1e2aa5 bytes) 000387[1] 1780808194.010453 write 0x726b48 thru 0x731170 (0xa629 bytes) 000388[1] 1780808194.029557 write 0x80f202 thru 0x818664 (0x9463 bytes) 000389[0] 1780808194.045134 write 0x1d5906 thru 0x1d6a2f (0x112a bytes) 000390[1] 1780808194.069243 trunc from 0x8e989a to 0x5f01dc (0x2f96bd bytes) 000393[1] 1780808194.121366 mapwrite 0x1d9c30 thru 0x1df1db (0x55ac bytes) 000396[1] 1780808194.184451 write 0x5bc2f9 thru 0x5c64fe (0xa206 bytes) 000397[0] 1780808194.203110 write 0x27fe72 thru 0x282318 (0x24a7 bytes) 000398[0] 1780808194.220993 trunc from 0x5f01dd to 0x4db515 (0x114cc7 bytes) 000400[0] 1780808194.241858 read 0x369ff4 thru 0x375bc9 (0xbbd6 bytes) 000402[0] 1780808194.263021 mapread 0x11fb78 thru 0x120599 (0xa22 bytes) 000403[1] 1780808194.270990 trunc from 0x4db516 to 0x07bbb6 (0x45f95f bytes) 000404[0] 1780808194.304128 read 0x064f74 thru 0x065cce (0xd5b bytes) 000406[1] 1780808194.327995 mapwrite 0x9fd9e8 thru 0x9fffff (0x2618 bytes) 000407[0] 1780808194.398019 read 0x13c5d3 thru 0x14b407 (0xee35 bytes) 000410[0] 1780808194.412359 mapread 0x168d7d thru 0x170fad (0x8231 bytes) 000411[1] 1780808194.431730 mapwrite 0x078e91 thru 0x084f49 (0xc0b9 bytes) 000414[1] 1780808194.514082 read 0x9a9cf4 thru 0x9a9e5a (0x167 bytes) 000416[0] 1780808194.540690 trunc from 0xa00000 to 0x92c65d (0xd39a2 bytes) 000420[0] 1780808194.588879 mapread 0x2dd6e6 thru 0x2e74a6 (0x9dc1 bytes) 000421[1] 1780808194.603725 write 0x7d823c thru 0x7dd92e (0x56f3 bytes) 000426[0] 1780808194.654654 write 0x5708c1 thru 0x57589c (0x4fdc bytes) 000427[1] 1780808194.670404 trunc from 0x92c65e to 0x02667f (0x905fde bytes) 000429[1] 1780808194.696329 read 0x006360 thru 0x010604 (0xa2a5 bytes) 000430[1] 1780808194.703061 mapread 0x00b534 thru 0x011b98 (0x6665 bytes) 000431[1] 1780808194.704005 mapread 0x021e1e thru 0x02667f (0x4862 bytes) 000433[0] 1780808194.713777 mapwrite 0x3414ca thru 0x342eb0 (0x19e7 bytes) 000434[1] 1780808194.794457 mapwrite 0x8fe781 thru 0x905d68 (0x75e8 bytes) 000435[0] 1780808194.928812 read 0x3380a3 thru 0x345e44 (0xdda2 bytes) 000436[1] 1780808194.986844 write 0x6eb759 thru 0x6ec680 (0xf28 bytes) 000439[0] 1780808194.991163 mapwrite 0x61567e thru 0x618425 (0x2da8 bytes) 000440[1] 1780808195.046705 read 0x1a567e thru 0x1b267c (0xcfff bytes) 000441[0] 1780808195.062088 read 0x05cbdb thru 0x06312c (0x6552 bytes) 000443[1] 1780808195.070608 mapwrite 0x7520e5 thru 0x75c81c (0xa738 bytes) 000445[1] 1780808195.145027 mapwrite 0x824baa thru 0x82da5a (0x8eb1 bytes) 000449[1] 1780808195.201520 read 0x004d44 thru 0x00de16 (0x90d3 bytes) 000453[0] 1780808195.215644 mapread 0x79f007 thru 0x7a73a1 (0x839b bytes) 000454[1] 1780808195.230745 trunc from 0x905d69 to 0x6763d1 (0x28f997 bytes) 000455[1] 1780808195.240575 mapwrite 0x6ee2ab thru 0x6f7cb0 (0x9a06 bytes) 000456[0] 1780808195.270023 mapwrite 0x154d41 thru 0x158cd3 (0x3f93 bytes) 000458[1] 1780808195.323025 mapwrite 0x08bee8 thru 0x09a7ae (0xe8c7 bytes) 000459[1] 1780808195.386519 trunc from 0x6f7cb1 to 0x2282eb (0x4cf9c5 bytes) 000460[1] 1780808195.398388 read 0x00fa10 thru 0x01086f (0xe60 bytes) 000463[1] 1780808195.421192 write 0x74b86a thru 0x74ee26 (0x35bd bytes) 000466[1] 1780808195.435058 trunc from 0x74ee27 to 0x90b2a9 (0x1bc483 bytes) 000467[0] 1780808195.453024 read 0x8ea476 thru 0x8efe40 (0x59cb bytes) 000469[0] 1780808195.467427 mapread 0x8c31ad thru 0x8d0eb1 (0xdd05 bytes) 000471[0] 1780808195.488050 mapread 0x192101 thru 0x199c9b (0x7b9b bytes) 000472[0] 1780808195.505338 trunc from 0x90b2aa to 0x2a8a0f (0x66289a bytes) 000475[0] 1780808195.542597 mapread 0x155fc2 thru 0x156b59 (0xb98 bytes) 000476[1] 1780808195.549456 write 0x87167a thru 0x880964 (0xf2eb bytes) 000479[1] 1780808195.584349 mapwrite 0x8702b4 thru 0x87c01a (0xbd67 bytes) 000482[1] 1780808195.640509 mapread 0x07b882 thru 0x0800e5 (0x4864 bytes) 000483[0] 1780808195.656177 mapwrite 0x613323 thru 0x61a0d5 (0x6db3 bytes) 000484[1] 1780808195.731375 mapwrite 0x467a88 thru 0x46d56f (0x5ae8 bytes) 000486[1] 1780808195.774672 write 0x9e2386 thru 0x9e9074 (0x6cef bytes) 000487[1] 1780808195.782756 mapwrite 0x80fd47 thru 0x81ad4d (0xb007 bytes) 000489[1] 1780808195.848449 mapwrite 0x8ebf88 thru 0x8f6e3a (0xaeb3 bytes) 000490[1] 1780808195.892521 trunc from 0x9e9075 to 0x5f14c7 (0x3f7bad bytes) 000493[0] 1780808195.917763 write 0x930cc4 thru 0x93a504 (0x9841 bytes) 000495[1] 1780808195.930634 mapread 0x580cdf thru 0x588684 (0x79a6 bytes) 000500[1] 1780808195.962859 mapread 0x3be33e thru 0x3c658e (0x8251 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 8196 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x3673cd 000002[1] 1780808196.303004 trunc from 00000000 to 0x3673cc (0x3673cd bytes) 000003[0] 1780808196.317740 mapread 0x11b000 thru 0x125c6f (0xac70 bytes) truncating to largest ever: 0x8d964b 000004[0] 1780808196.341296 trunc from 0x3673cd to 0x8d964a (0x57227e bytes) 000009[1] 1780808196.372164 write 0x7ca000 thru 0x7d9cb2 (0xfcb3 bytes) 000010[0] 1780808196.378739 read 0x60e000 thru 0x618471 (0xa472 bytes) 000012[1] 1780808196.395019 mapwrite 0x0da000 thru 0x0e1178 (0x7179 bytes) 000016[0] 1780808196.466045 write 0x69c000 thru 0x6a7a15 (0xba16 bytes) 000017[1] 1780808196.477442 read 0x604000 thru 0x613403 (0xf404 bytes) 000019[0] 1780808196.488543 read 0x2c1000 thru 0x2c576e (0x476f bytes) truncating to largest ever: 0x902d5e 000020[0] 1780808196.491828 trunc from 0x8d964b to 0x902d5d (0x29713 bytes) 000023[1] 1780808196.509050 trunc from 0x902d5e to 0x243ef6 (0x6bee67 bytes) 000025[0] 1780808196.525140 read 0x238000 thru 0x2432eb (0xb2ec bytes) 000026[0] 1780808196.541674 write 0x6bd000 thru 0x6bedb9 (0x1dba bytes) 000027[0] 1780808196.553527 mapwrite 0x109000 thru 0x10eac1 (0x5ac2 bytes) 000029[0] 1780808196.589386 mapread 0x669000 thru 0x672b82 (0x9b83 bytes) 000032[0] 1780808196.613500 mapread 0x167000 thru 0x16a32f (0x3330 bytes) 000034[0] 1780808196.629598 write 0x9a9000 thru 0x9b6573 (0xd574 bytes) 000035[0] 1780808196.638457 trunc from 0x9b6574 to 0x4cd063 (0x4e9510 bytes) 000036[1] 1780808196.661816 read 0x13f000 thru 0x143089 (0x408a bytes) 000037[1] 1780808196.687662 trunc from 0x4cd064 to 0x28c819 (0x24084a bytes) 000042[0] 1780808196.727573 trunc from 0x28c81a to 0x537369 (0x2aab50 bytes) 000048[0] 1780808196.794878 mapread 0x10b000 thru 0x10d64f (0x2650 bytes) 000049[1] 1780808196.812647 mapwrite 0x344000 thru 0x34e8ee (0xa8ef bytes) 000052[1] 1780808196.852604 trunc from 0x53736a to 0x02b500 (0x50be69 bytes) 000053[1] 1780808196.862674 read 0x018000 thru 0x018a62 (0xa63 bytes) 000055[0] 1780808196.878056 mapwrite 0x639000 thru 0x63a5ca (0x15cb bytes) 000056[1] 1780808196.926048 mapread 0x16d000 thru 0x1754a6 (0x84a7 bytes) 000058[0] 1780808196.970704 mapread 0x505000 thru 0x50f18d (0xa18e bytes) 000066[1] 1780808196.997847 mapwrite 0x105000 thru 0x10be27 (0x6e28 bytes) 000068[0] 1780808197.084920 mapwrite 0x816000 thru 0x81c429 (0x642a bytes) 000071[0] 1780808197.164115 read 0x290000 thru 0x29a7ec (0xa7ed bytes) 000074[1] 1780808197.176162 read 0x2b5000 thru 0x2b9cb6 (0x4cb7 bytes) 000076[0] 1780808197.186621 read 0x2e8000 thru 0x2eda9f (0x5aa0 bytes) 000077[0] 1780808197.190520 trunc from 0x81c42a to 0x74116e (0xdb2bb bytes) 000078[1] 1780808197.205510 write 0x3ed000 thru 0x3ee139 (0x113a bytes) 000082[1] 1780808197.234944 mapread 0x62f000 thru 0x63e022 (0xf023 bytes) 000083[0] 1780808197.253848 read 0x0d6000 thru 0x0e31fe (0xd1ff bytes) 000084[0] 1780808197.261081 write 0x53d000 thru 0x54c1c3 (0xf1c4 bytes) 000086[1] 1780808197.270039 mapwrite 0x2f5000 thru 0x302dc2 (0xddc3 bytes) 000087[0] 1780808197.313538 mapread 0x328000 thru 0x32a204 (0x2205 bytes) 000088[0] 1780808197.332106 mapread 0x2d5000 thru 0x2dac49 (0x5c4a bytes) 000089[1] 1780808197.336839 write 0x9f9000 thru 0x9ff92e (0x692f bytes) 000090[0] 1780808197.344588 mapwrite 0x417000 thru 0x422589 (0xb58a bytes) 000092[0] 1780808197.388604 mapwrite 0x65d000 thru 0x66557e (0x857f bytes) 000093[1] 1780808197.413824 trunc from 0x9ff92f to 0x750eb6 (0x2aea78 bytes) 000094[0] 1780808197.433639 mapwrite 0x9ee000 thru 0x9fdff9 (0xfffa bytes) 000096[1] 1780808197.478094 read 0x124000 thru 0x12a678 (0x6679 bytes) 000099[1] 1780808197.491503 mapwrite 0x544000 thru 0x548b80 (0x4b81 bytes) 000100[1] 1780808197.535786 mapread 0x43d000 thru 0x44c8c2 (0xf8c3 bytes) 000101[1] 1780808197.547643 trunc from 0x9fdffa to 0x54fad6 (0x4ae523 bytes) 000105[0] 1780808197.572558 read 0x4b0000 thru 0x4b8e08 (0x8e09 bytes) 000106[1] 1780808197.586657 mapread 0x4c7000 thru 0x4cb11d (0x411e bytes) 000109[1] 1780808197.592165 write 0x826000 thru 0x8298ce (0x38cf bytes) 000113[0] 1780808197.611679 trunc from 0x8298cf to 0x277b33 (0x5b1d9b bytes) 000115[1] 1780808197.628901 read 0x16d000 thru 0x17a85a (0xd85b bytes) 000120[0] 1780808197.660040 read 0x028000 thru 0x02e687 (0x6688 bytes) 000121[0] 1780808197.683319 trunc from 0x277b34 to 0x75b4b1 (0x4e397e bytes) 000123[1] 1780808197.705153 write 0x1c7000 thru 0x1d3c03 (0xcc04 bytes) 000126[1] 1780808197.729332 mapwrite 0x45e000 thru 0x46d0d8 (0xf0d9 bytes) 000127[1] 1780808197.771565 read 0x592000 thru 0x593109 (0x110a bytes) truncating to largest ever: 0x95b60d 000129[0] 1780808197.785426 trunc from 0x75b4b2 to 0x95b60c (0x20015b bytes) 000132[1] 1780808197.809438 mapread 0x42c000 thru 0x438bef (0xcbf0 bytes) 000133[0] 1780808197.824701 write 0x794000 thru 0x799d75 (0x5d76 bytes) 000136[1] 1780808197.856899 mapwrite 0x1fd000 thru 0x1fddcf (0xdd0 bytes) 000137[1] 1780808197.888618 write 0x9fd000 thru 0x9ff19a (0x219b bytes) 000138[0] 1780808197.894000 write 0x5a9000 thru 0x5aaed1 (0x1ed2 bytes) 000141[1] 1780808197.909645 read 0x216000 thru 0x224b55 (0xeb56 bytes) 000143[0] 1780808197.917945 read 0x327000 thru 0x331658 (0xa659 bytes) 000145[1] 1780808197.921126 mapwrite 0x581000 thru 0x5866e3 (0x56e4 bytes) 000147[0] 1780808197.948405 write 0x186000 thru 0x194399 (0xe39a bytes) 000149[1] 1780808197.958699 trunc from 0x9ff19b to 0x09fa3d (0x95f75d bytes) 000151[1] 1780808197.981011 trunc from 0x09fa3e to 0x6850e1 (0x5e56a4 bytes) 000152[0] 1780808197.994902 write 0x478000 thru 0x4836be (0xb6bf bytes) 000155[1] 1780808198.008851 trunc from 0x6850e2 to 0x277f98 (0x40d149 bytes) 000157[0] 1780808198.022435 mapread 0x149000 thru 0x154f91 (0xbf92 bytes) 000159[1] 1780808198.046030 read 0x194000 thru 0x19b502 (0x7503 bytes) 000162[0] 1780808198.067731 read 0x110000 thru 0x118bf1 (0x8bf2 bytes) 000163[1] 1780808198.084687 trunc from 0x277f99 to 0x5f57f2 (0x37d85a bytes) 000164[0] 1780808198.101393 trunc from 0x5f57f3 to 0x52e7a7 (0xc704b bytes) 000165[0] 1780808198.114278 trunc from 0x52e7a8 to 0x27564e (0x2b9159 bytes) 000166[1] 1780808198.123906 mapwrite 0x769000 thru 0x7717fe (0x87ff bytes) 000168[0] 1780808198.186682 mapread 0x4f0000 thru 0x4fe65b (0xe65c bytes) 000173[1] 1780808198.218782 write 0x081000 thru 0x089096 (0x8097 bytes) 000174[1] 1780808198.233167 write 0x3a6000 thru 0x3b13cb (0xb3cc bytes) 000175[0] 1780808198.241057 mapread 0x758000 thru 0x766a9e (0xea9f bytes) 000177[0] 1780808198.260956 write 0x2bc000 thru 0x2c74f1 (0xb4f2 bytes) 000181[1] 1780808198.284493 trunc from 0x7717ff to 0x298c49 (0x4d8bb5 bytes) 000183[0] 1780808198.301134 write 0x90d000 thru 0x91929c (0xc29d bytes) 000184[0] 1780808198.311713 mapwrite 0x8d9000 thru 0x8e3460 (0xa461 bytes) 000186[1] 1780808198.377198 trunc from 0x91929d to 0x900d26 (0x18576 bytes) 000187[1] 1780808198.394082 trunc from 0x900d27 to 0x8eaf9c (0x15d8a bytes) 000188[0] 1780808198.413530 mapread 0x49f000 thru 0x4ad52e (0xe52f bytes) 000189[0] 1780808198.444596 mapread 0x489000 thru 0x493c75 (0xac76 bytes) 000193[0] 1780808198.475176 read 0x7c8000 thru 0x7cabba (0x2bbb bytes) 000196[0] 1780808198.482459 write 0x301000 thru 0x3047db (0x37dc bytes) 000201[0] 1780808198.502885 write 0x6e4000 thru 0x6ecd59 (0x8d5a bytes) 000204[1] 1780808198.525421 write 0x7db000 thru 0x7e2b2a (0x7b2b bytes) 000205[0] 1780808198.558739 mapwrite 0x09c000 thru 0x0a7a60 (0xba61 bytes) 000206[0] 1780808198.607408 write 0x250000 thru 0x258ee0 (0x8ee1 bytes) 000208[0] 1780808198.611418 trunc from 0x8eaf9d to 0x72165c (0x1c9940 bytes) 000213[1] 1780808198.631743 mapread 0x2a2000 thru 0x2a4629 (0x262a bytes) 000215[0] 1780808198.645135 read 0x00b000 thru 0x00c020 (0x1021 bytes) 000217[1] 1780808198.648859 mapwrite 0x88f000 thru 0x89bcb8 (0xccb9 bytes) 000219[1] 1780808198.693314 trunc from 0x89bcb9 to 0x440469 (0x45b84f bytes) 000220[1] 1780808198.714406 read 0x091000 thru 0x09345b (0x245c bytes) truncating to largest ever: 0x9f7977 000222[0] 1780808198.733159 trunc from 0x44046a to 0x9f7976 (0x5b750d bytes) 000225[0] 1780808198.758821 mapwrite 0x977000 thru 0x97b362 (0x4363 bytes) 000226[1] 1780808198.798598 mapread 0x55c000 thru 0x569e96 (0xde97 bytes) 000228[1] 1780808198.819452 mapwrite 0x7b9000 thru 0x7c6491 (0xd492 bytes) 000229[1] 1780808198.856928 mapwrite 0x13e000 thru 0x14accc (0xcccd bytes) 000231[0] 1780808198.896774 read 0x8b2000 thru 0x8b8e5d (0x6e5e bytes) 000232[0] 1780808198.903701 mapwrite 0x10b000 thru 0x10c081 (0x1082 bytes) 000233[1] 1780808198.924010 mapwrite 0x080000 thru 0x080af9 (0xafa bytes) 000234[1] 1780808198.945982 read 0x1ef000 thru 0x1f156a (0x256b bytes) 000237[1] 1780808198.954076 write 0x1a8000 thru 0x1b5a69 (0xda6a bytes) 000241[0] 1780808198.966065 read 0x8c3000 thru 0x8d0d90 (0xdd91 bytes) 000242[0] 1780808198.978392 trunc from 0x9f7977 to 0x84aa58 (0x1acf1e bytes) 000243[0] 1780808198.990015 mapwrite 0x47d000 thru 0x486e1e (0x9e1f bytes) 000245[0] 1780808199.024162 mapread 0x4e3000 thru 0x4ea4c2 (0x74c3 bytes) 000246[1] 1780808199.028341 read 0x485000 thru 0x48c470 (0x7471 bytes) 000247[0] 1780808199.037235 mapwrite 0x5d7000 thru 0x5dc527 (0x5528 bytes) 000249[1] 1780808199.082681 read 0x0d2000 thru 0x0d81c0 (0x61c1 bytes) 000251[0] 1780808199.102906 trunc from 0x84aa59 to 0x79dfd1 (0xaca87 bytes) 000258[1] 1780808199.158944 mapread 0x320000 thru 0x32998d (0x998e bytes) 000259[0] 1780808199.177460 mapwrite 0x8b6000 thru 0x8c4e87 (0xee88 bytes) 000260[1] 1780808199.239674 write 0x2fb000 thru 0x2fed1c (0x3d1d bytes) 000261[0] 1780808199.252105 write 0x4d6000 thru 0x4e35f4 (0xd5f5 bytes) 000262[0] 1780808199.260138 trunc from 0x8c4e88 to 0x162f7d (0x761f0a bytes) 000263[1] 1780808199.264798 write 0x898000 thru 0x89bda1 (0x3da2 bytes) 000264[0] 1780808199.277326 mapwrite 0x9ab000 thru 0x9b8fa9 (0xdfaa bytes) 000265[1] 1780808199.329801 mapread 0x277000 thru 0x27e580 (0x7581 bytes) 000269[0] 1780808199.342455 write 0x468000 thru 0x468631 (0x632 bytes) 000270[0] 1780808199.352774 write 0x513000 thru 0x5216cb (0xe6cc bytes) 000271[1] 1780808199.365183 write 0x4e1000 thru 0x4e9826 (0x8827 bytes) 000276[1] 1780808199.395382 read 0x75e000 thru 0x768c60 (0xac61 bytes) 000278[1] 1780808199.404175 write 0x4f5000 thru 0x5041ae (0xf1af bytes) 000279[1] 1780808199.413537 mapread 0x8e5000 thru 0x8e6947 (0x1948 bytes) 000280[0] 1780808199.419044 mapread 0x494000 thru 0x495b11 (0x1b12 bytes) 000283[1] 1780808199.440943 trunc from 0x9b8faa to 0x6dc2d5 (0x2dccd4 bytes) 000284[1] 1780808199.459582 mapread 0x6af000 thru 0x6b583a (0x683b bytes) 000287[0] 1780808199.492849 read 0x4e2000 thru 0x4e93af (0x73b0 bytes) 000288[0] 1780808199.501559 mapwrite 0x1b0000 thru 0x1b2e09 (0x2e0a bytes) 000291[1] 1780808199.548505 trunc from 0x6dc2d6 to 0x49038f (0x24bf46 bytes) 000292[1] 1780808199.561642 trunc from 0x490390 to 0x509a99 (0x7970a bytes) 000294[0] 1780808199.569810 write 0x3f1000 thru 0x3f2400 (0x1401 bytes) 000295[1] 1780808199.581781 read 0x311000 thru 0x31ee1b (0xde1c bytes) 000296[1] 1780808199.601005 trunc from 0x509a9a to 0x109e4f (0x3ffc4a bytes) 000297[0] 1780808199.625756 mapread 0x05a000 thru 0x05f792 (0x5793 bytes) 000301[0] 1780808199.654341 write 0x902000 thru 0x902527 (0x528 bytes) 000304[1] 1780808199.684695 write 0x781000 thru 0x78bd53 (0xad54 bytes) 000305[1] 1780808199.691723 trunc from 0x902528 to 0x5ae2fd (0x35422a bytes) 000306[0] 1780808199.699067 write 0x0fc000 thru 0x108c49 (0xcc4a bytes) 000310[0] 1780808199.713688 mapwrite 0x94a000 thru 0x94a2f7 (0x2f8 bytes) 000313[1] 1780808199.759382 mapread 0x10c000 thru 0x1139ae (0x79af bytes) 000315[0] 1780808199.781288 mapread 0x418000 thru 0x427bea (0xfbeb bytes) 000316[0] 1780808199.799867 mapread 0x2d5000 thru 0x2d7007 (0x2008 bytes) 000321[0] 1780808199.810929 mapwrite 0x2ab000 thru 0x2b6e64 (0xbe65 bytes) 000322[0] 1780808199.845978 read 0x89e000 thru 0x8a9316 (0xb317 bytes) 000325[0] 1780808199.877480 write 0x3fa000 thru 0x403677 (0x9678 bytes) 000326[0] 1780808199.895979 read 0x312000 thru 0x31212f (0x130 bytes) 000327[0] 1780808199.899447 mapread 0x7f4000 thru 0x7ff1e0 (0xb1e1 bytes) 000329[0] 1780808199.917867 mapwrite 0x7ec000 thru 0x7fb96b (0xf96c bytes) 000331[0] 1780808199.951610 write 0x788000 thru 0x792d86 (0xad87 bytes) 000333[1] 1780808199.955848 mapwrite 0x8a9000 thru 0x8b45cd (0xb5ce bytes) 000334[1] 1780808200.016797 read 0x296000 thru 0x29a074 (0x4075 bytes) 000337[1] 1780808200.039359 trunc from 0x94a2f8 to 0x28eee6 (0x6bb411 bytes) 000339[0] 1780808200.053927 read 0x11d000 thru 0x11f67a (0x267b bytes) 000340[0] 1780808200.073510 trunc from 0x28eee7 to 0x1a6616 (0xe88d0 bytes) 000341[1] 1780808200.091735 mapwrite 0x4b8000 thru 0x4c49bc (0xc9bd bytes) 000342[1] 1780808200.137853 mapread 0x3b0000 thru 0x3b67ae (0x67af bytes) 000343[0] 1780808200.155440 mapread 0x065000 thru 0x069554 (0x4555 bytes) 000344[1] 1780808200.189107 mapread 0x4bc000 thru 0x4bd00a (0x100b bytes) 000345[0] 1780808200.194849 trunc from 0x4c49bd to 0x2917de (0x2331de bytes) 000350[0] 1780808200.230865 mapread 0x1c1000 thru 0x1c3102 (0x2103 bytes) 000354[0] 1780808200.249423 write 0x7cc000 thru 0x7cf07e (0x307f bytes) 000355[0] 1780808200.252201 trunc from 0x7cf07f to 0x6af2a0 (0x11fdde bytes) 000356[0] 1780808200.271390 write 0x71a000 thru 0x72763b (0xd63c bytes) 000358[1] 1780808200.296416 write 0x13f000 thru 0x14b52d (0xc52e bytes) 000362[1] 1780808200.347603 trunc from 0x72763c to 0x590420 (0x19721b bytes) 000368[0] 1780808200.377501 trunc from 0x590421 to 0x17d7ab (0x412c75 bytes) 000371[0] 1780808200.421326 trunc from 0x17d7ac to 0x224b0a (0xa735f bytes) 000374[1] 1780808200.442041 mapwrite 0x180000 thru 0x186f36 (0x6f37 bytes) 000377[0] 1780808200.496516 trunc from 0x224b0b to 0x29c762 (0x77c58 bytes) 000378[0] 1780808200.516818 mapwrite 0x0cc000 thru 0x0da3ad (0xe3ae bytes) 000381[1] 1780808200.629334 mapread 0x25a000 thru 0x264b87 (0xab88 bytes) 000383[1] 1780808200.665423 mapwrite 0x88d000 thru 0x88e0ac (0x10ad bytes) 000387[1] 1780808200.726048 trunc from 0x88e0ad to 0x8124b9 (0x7bbf3 bytes) 000388[0] 1780808200.732237 mapwrite 0x01d000 thru 0x028941 (0xb942 bytes) 000389[1] 1780808200.784410 mapread 0x211000 thru 0x21ecf9 (0xdcfa bytes) 000390[1] 1780808200.818551 write 0x9de000 thru 0x9e5693 (0x7694 bytes) 000391[1] 1780808200.832357 write 0x45a000 thru 0x45a3b6 (0x3b7 bytes) 000393[1] 1780808200.845302 mapwrite 0x6e3000 thru 0x6e3fe6 (0xfe7 bytes) 000394[1] 1780808200.871312 trunc from 0x9e5694 to 0x70a09f (0x2db5f4 bytes) 000395[0] 1780808200.892571 write 0x6cc000 thru 0x6cf295 (0x3296 bytes) 000397[0] 1780808200.931797 trunc from 0x70a0a0 to 0x50d1a7 (0x1fcef8 bytes) 000399[0] 1780808200.958546 write 0x0a9000 thru 0x0af2c9 (0x62ca bytes) 000400[1] 1780808200.966122 write 0x89c000 thru 0x8a9344 (0xd345 bytes) 000405[0] 1780808201.000917 mapwrite 0x88b000 thru 0x8906bc (0x56bd bytes) 000406[0] 1780808201.075231 mapread 0x874000 thru 0x87b5b0 (0x75b1 bytes) 000408[1] 1780808201.090047 write 0x014000 thru 0x01a7be (0x67bf bytes) 000409[0] 1780808201.115824 write 0x9d6000 thru 0x9e133a (0xb33b bytes) 000412[0] 1780808201.161954 mapread 0x564000 thru 0x564663 (0x664 bytes) 000413[0] 1780808201.169579 trunc from 0x9e133b to 0x242998 (0x79e9a2 bytes) 000414[1] 1780808201.188590 read 0x1e4000 thru 0x1f35f7 (0xf5f8 bytes) 000415[0] 1780808201.214375 trunc from 0x242999 to 0x6ffd55 (0x4bd3bd bytes) 000416[0] 1780808201.248429 read 0x37c000 thru 0x385330 (0x9331 bytes) 000418[0] 1780808201.276233 read 0x179000 thru 0x186d90 (0xdd91 bytes) 000420[1] 1780808201.295376 mapread 0x396000 thru 0x3a297e (0xc97f bytes) 000423[0] 1780808201.345580 read 0x4f6000 thru 0x4ff622 (0x9623 bytes) 000424[1] 1780808201.353869 mapwrite 0x995000 thru 0x99d107 (0x8108 bytes) 000425[0] 1780808201.414329 trunc from 0x99d108 to 0x4cac00 (0x4d2507 bytes) 000427[0] 1780808201.435335 mapread 0x0a5000 thru 0x0b3721 (0xe722 bytes) 000428[1] 1780808201.447630 write 0x822000 thru 0x82e2ef (0xc2f0 bytes) 000429[0] 1780808201.453947 trunc from 0x82e2f0 to 0x416950 (0x41799f bytes) 000432[0] 1780808201.491636 mapwrite 0x80a000 thru 0x81214b (0x814c bytes) 000434[1] 1780808201.526185 mapwrite 0x217000 thru 0x21ccae (0x5caf bytes) 000435[0] 1780808201.562011 mapread 0x55b000 thru 0x5671a1 (0xc1a2 bytes) 000438[1] 1780808201.578978 write 0x4e3000 thru 0x4f112b (0xe12c bytes) 000440[1] 1780808201.590740 write 0x80e000 thru 0x80ecb3 (0xcb4 bytes) 000442[0] 1780808201.597943 mapread 0x690000 thru 0x697902 (0x7903 bytes) 000443[0] 1780808201.612987 mapwrite 0x608000 thru 0x616b2e (0xeb2f bytes) 000445[1] 1780808201.659124 read 0x229000 thru 0x22dd7e (0x4d7f bytes) 000447[0] 1780808201.672671 trunc from 0x81214c to 0x3958c2 (0x47c889 bytes) 000448[1] 1780808201.687962 trunc from 0x3958c3 to 0x4e7f41 (0x15267f bytes) 000450[1] 1780808201.701909 mapwrite 0x909000 thru 0x913cf7 (0xacf8 bytes) 000455[0] 1780808201.765819 read 0x62b000 thru 0x62f973 (0x4974 bytes) 000456[0] 1780808201.778587 write 0x85b000 thru 0x86217d (0x717e bytes) 000461[0] 1780808201.807758 trunc from 0x913cf8 to 0x86c2dd (0xa7a1a bytes) 000463[1] 1780808201.837431 trunc from 0x86c2de to 0x6753ca (0x1f6f13 bytes) 000464[1] 1780808201.850182 mapread 0x647000 thru 0x653dce (0xcdcf bytes) 000465[0] 1780808201.869483 mapread 0x1a6000 thru 0x1ac2ae (0x62af bytes) 000469[0] 1780808201.886972 mapwrite 0x5cd000 thru 0x5d4b4f (0x7b50 bytes) 000470[0] 1780808201.916169 mapwrite 0x273000 thru 0x27b529 (0x852a bytes) 000472[1] 1780808201.943489 mapread 0x189000 thru 0x18f530 (0x6531 bytes) 000476[1] 1780808201.957582 write 0x3c9000 thru 0x3d7ddc (0xeddd bytes) 000477[0] 1780808201.965764 write 0x933000 thru 0x937937 (0x4938 bytes) 000481[1] 1780808201.983724 mapwrite 0x1d0000 thru 0x1d4dc4 (0x4dc5 bytes) 000483[1] 1780808202.031203 read 0x098000 thru 0x0a6103 (0xe104 bytes) 000486[1] 1780808202.048652 read 0x1d3000 thru 0x1d903a (0x603b bytes) 000487[0] 1780808202.049131 mapread 0x0a9000 thru 0x0b4371 (0xb372 bytes) 000488[1] 1780808202.064514 mapread 0x715000 thru 0x71a0c7 (0x50c8 bytes) 000490[0] 1780808202.079492 mapread 0x014000 thru 0x01c6f3 (0x86f4 bytes) 000492[0] 1780808202.085784 read 0x19d000 thru 0x1a9b09 (0xcb0a bytes) 000493[0] 1780808202.091803 read 0x443000 thru 0x44c41b (0x941c bytes) 000494[0] 1780808202.096675 mapread 0x2a3000 thru 0x2a546c (0x246d bytes) 000495[0] 1780808202.108828 mapread 0x40b000 thru 0x4131eb (0x81ec bytes) 000500[0] 1780808202.117702 mapread 0x00c000 thru 0x018720 (0xc721 bytes) All operations completed A-OK! PASS 16a (18s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 00:56:44 (1780808204) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.159312 s, 52.7 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.156462 s, 53.6 MB/s Chance of close/open is 1 in 50 Seed set to 8205 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1780808205.616494 mapwrite 0x7e034b thru 0x7e18df (0x1595 bytes) 000002[1] 1780808205.657972 mapwrite 0x6dfa89 thru 0x6e0c5c (0x11d4 bytes) 000004[1] 1780808205.717325 write 0x20b441 thru 0x20b60b (0x1cb bytes) truncating to largest ever: 0x71b3ea 000006[0] 1780808205.749558 trunc from 0x7e18e0 to 0x71b3e9 (0xc64f6 bytes) 000007[1] 1780808205.793016 read 0x1fd0ba thru 0x1fd686 (0x5cd bytes) 000010[0] 1780808205.817146 mapwrite 0x6c5333 thru 0x6c676a (0x1438 bytes) 000013[1] 1780808205.860024 mapwrite 0x127622 thru 0x12855c (0xf3b bytes) 000015[0] 1780808205.947387 read 0x3b53fb thru 0x3b67e0 (0x13e6 bytes) 000016[0] 1780808205.958858 mapwrite 0x9a4560 thru 0x9a6551 (0x1ff2 bytes) 000018[1] 1780808206.039330 mapread 0x25c106 thru 0x25dcfe (0x1bf9 bytes) 000020[0] 1780808206.056926 mapread 0x0e8126 thru 0x0e867d (0x558 bytes) 000022[1] 1780808206.068328 write 0x415905 thru 0x415dd8 (0x4d4 bytes) 000026[0] 1780808206.104026 trunc from 0x9a6552 to 0x5c1c87 (0x3e48ca bytes) 000027[1] 1780808206.144519 write 0x1d73dc thru 0x1d8536 (0x115b bytes) 000033[1] 1780808206.227176 mapread 0x083a43 thru 0x0848c4 (0xe82 bytes) 000034[0] 1780808206.269150 write 0x0af39d thru 0x0b08d1 (0x1535 bytes) 000035[0] 1780808206.329494 mapread 0x077458 thru 0x078972 (0x151b bytes) 000036[1] 1780808206.355885 read 0x4a4fc7 thru 0x4a4ffc (0x36 bytes) 000037[0] 1780808206.375654 trunc from 0x5c1c88 to 0x389321 (0x238966 bytes) 000038[0] 1780808206.387119 trunc from 0x389322 to 0x22533a (0x163fe7 bytes) 000040[0] 1780808206.433786 write 0x5116eb thru 0x512b17 (0x142d bytes) 000043[0] 1780808206.454532 trunc from 0x512b18 to 0x536ed7 (0x243c0 bytes) 000045[1] 1780808206.486497 read 0x312626 thru 0x31452c (0x1f07 bytes) 000049[1] 1780808206.506886 trunc from 0x536ed8 to 0x56f110 (0x38239 bytes) 000050[0] 1780808206.540846 mapwrite 0x55c465 thru 0x55d188 (0xd24 bytes) 000051[1] 1780808206.567705 write 0x6c1a5d thru 0x6c1d5d (0x301 bytes) 000052[1] 1780808206.571745 trunc from 0x6c1d5e to 0x3aabe2 (0x31717b bytes) 000053[0] 1780808206.586117 mapwrite 0x6fdd73 thru 0x6fe7cf (0xa5d bytes) 000054[0] 1780808206.649425 write 0x43afd6 thru 0x43beed (0xf18 bytes) 000055[1] 1780808206.660047 mapwrite 0x93af30 thru 0x93b396 (0x467 bytes) 000058[0] 1780808206.727496 mapread 0x153f22 thru 0x1541ed (0x2cc bytes) 000060[1] 1780808206.744116 mapwrite 0x202f92 thru 0x203e8e (0xefd bytes) 000061[1] 1780808206.806888 read 0x5e3527 thru 0x5e530f (0x1de9 bytes) 000063[0] 1780808206.826047 trunc from 0x93b397 to 0x0fb757 (0x83fc3f bytes) 000064[0] 1780808206.864323 mapwrite 0x605694 thru 0x606c37 (0x15a4 bytes) 000065[0] 1780808206.941700 read 0x28dc09 thru 0x28faa6 (0x1e9e bytes) 000067[1] 1780808206.982217 trunc from 0x606c38 to 0x387073 (0x27fbc4 bytes) 000068[1] 1780808207.011555 mapwrite 0x1949fa thru 0x1950cb (0x6d2 bytes) 000070[1] 1780808207.040332 mapread 0x021409 thru 0x021863 (0x45b bytes) 000072[1] 1780808207.065633 mapwrite 0x15ef66 thru 0x15fa33 (0xace bytes) 000076[1] 1780808207.183352 mapwrite 0x21fd47 thru 0x221320 (0x15da bytes) 000078[0] 1780808207.247032 mapread 0x326041 thru 0x32713f (0x10ff bytes) 000080[1] 1780808207.295222 read 0x011253 thru 0x011b70 (0x91e bytes) 000085[0] 1780808207.352484 mapread 0x0c8e99 thru 0x0c9148 (0x2b0 bytes) 000086[0] 1780808207.369866 trunc from 0x387074 to 0x2ff55c (0x87b17 bytes) 000090[0] 1780808207.456795 mapread 0x1a693b thru 0x1a745a (0xb20 bytes) 000094[0] 1780808207.505187 read 0x17367f thru 0x173dc2 (0x744 bytes) 000095[0] 1780808207.575954 read 0x1f21d2 thru 0x1f2e40 (0xc6f bytes) 000096[1] 1780808207.576790 write 0x44de47 thru 0x44e91a (0xad4 bytes) 000098[0] 1780808207.611931 mapwrite 0x690549 thru 0x692297 (0x1d4f bytes) 000099[0] 1780808207.720285 write 0x75dc5f thru 0x75f410 (0x17b2 bytes) 000103[1] 1780808207.742369 trunc from 0x75f411 to 0x111a42 (0x64d9ce bytes) 000105[0] 1780808207.788651 read 0x0976cf thru 0x098f61 (0x1893 bytes) 000107[0] 1780808207.845878 mapwrite 0x57e924 thru 0x57fb42 (0x121f bytes) truncating to largest ever: 0x9b61ef 000108[1] 1780808207.943425 trunc from 0x57fb43 to 0x9b61ee (0x4366ac bytes) 000109[1] 1780808207.960782 mapwrite 0x45e4a3 thru 0x45e816 (0x374 bytes) 000110[0] 1780808208.037180 write 0x180b9f thru 0x18123e (0x6a0 bytes) 000112[1] 1780808208.078275 read 0x34815b thru 0x3484f1 (0x397 bytes) truncating to largest ever: 0x9bed2a 000115[1] 1780808208.098624 trunc from 0x9b61ef to 0x9bed29 (0x8b3b bytes) 000117[1] 1780808208.119404 write 0x02b29f thru 0x02ca97 (0x17f9 bytes) 000118[1] 1780808208.124840 trunc from 0x9bed2a to 0x0aa883 (0x9144a6 bytes) 000119[0] 1780808208.138895 read 0x041c5d thru 0x043557 (0x18fb bytes) 000121[1] 1780808208.174401 trunc from 0x0aa884 to 0x8cbcd3 (0x821450 bytes) 000124[0] 1780808208.227181 mapread 0x7a6651 thru 0x7a7922 (0x12d2 bytes) 000126[1] 1780808208.269461 mapread 0x7d3ced thru 0x7d593c (0x1c50 bytes) 000128[1] 1780808208.300456 trunc from 0x8cbcd4 to 0x96a885 (0x9ebb2 bytes) 000130[1] 1780808208.345339 write 0x5a1845 thru 0x5a3655 (0x1e11 bytes) 000132[0] 1780808208.386105 write 0x780c50 thru 0x781e95 (0x1246 bytes) 000133[0] 1780808208.419134 mapwrite 0x848f39 thru 0x84a047 (0x110f bytes) 000135[0] 1780808208.461591 mapwrite 0x1df259 thru 0x1e0c8a (0x1a32 bytes) 000136[0] 1780808208.489997 read 0x47f98a thru 0x480abf (0x1136 bytes) 000137[0] 1780808208.497901 write 0x3f9f7f thru 0x3fb44d (0x14cf bytes) 000140[0] 1780808208.521180 mapread 0x4259f7 thru 0x4265ed (0xbf7 bytes) 000142[0] 1780808208.544133 read 0x1cb37f thru 0x1cbffe (0xc80 bytes) 000143[1] 1780808208.559860 mapwrite 0x7d1258 thru 0x7d2fbe (0x1d67 bytes) 000144[0] 1780808208.612784 read 0x0a1fd8 thru 0x0a3004 (0x102d bytes) 000147[0] 1780808208.641583 read 0x70eef3 thru 0x71023c (0x134a bytes) 000148[1] 1780808208.666321 read 0x2e4d17 thru 0x2e6c04 (0x1eee bytes) 000151[0] 1780808208.686472 write 0x42a43e thru 0x42bd26 (0x18e9 bytes) 000152[1] 1780808208.708861 mapwrite 0x56845d thru 0x568d40 (0x8e4 bytes) 000154[0] 1780808208.746397 trunc from 0x96a886 to 0x029167 (0x94171e bytes) 000155[0] 1780808208.762042 mapread 0x01ea35 thru 0x020377 (0x1943 bytes) 000158[1] 1780808208.784641 mapread 0x02682d thru 0x02762a (0xdfe bytes) 000159[0] 1780808208.811408 mapread 0x01b288 thru 0x01b922 (0x69b bytes) 000160[1] 1780808208.822637 read 0x003113 thru 0x004e47 (0x1d35 bytes) 000161[1] 1780808208.829130 read 0x001095 thru 0x002602 (0x156e bytes) 000162[0] 1780808208.841658 mapread 0x022814 thru 0x0245f4 (0x1de1 bytes) 000163[1] 1780808208.850523 read 0x000f67 thru 0x00235d (0x13f7 bytes) 000168[1] 1780808208.856952 read 0x029148 thru 0x029167 (0x20 bytes) 000169[0] 1780808208.861465 read 0x0151ed thru 0x016c68 (0x1a7c bytes) 000171[1] 1780808208.866245 mapread 0x00e950 thru 0x010699 (0x1d4a bytes) 000173[0] 1780808208.869187 read 0x009815 thru 0x009a45 (0x231 bytes) 000174[1] 1780808208.872590 mapread 0x016a48 thru 0x017804 (0xdbd bytes) 000175[1] 1780808208.875456 mapwrite 0x54070d thru 0x541a21 (0x1315 bytes) 000176[1] 1780808208.936161 write 0x96bfb7 thru 0x96d93d (0x1987 bytes) 000177[0] 1780808208.939880 write 0x104e7f thru 0x1050ff (0x281 bytes) 000178[1] 1780808209.002832 trunc from 0x96d93e to 0x643def (0x329b4e bytes) 000181[1] 1780808209.036764 write 0x0a7a45 thru 0x0a8532 (0xaee bytes) 000182[0] 1780808209.046928 write 0x7cf833 thru 0x7d0371 (0xb3f bytes) 000183[1] 1780808209.060039 read 0x2354ae thru 0x236d7d (0x18d0 bytes) 000186[1] 1780808209.079934 write 0x1b0fef thru 0x1b2273 (0x1285 bytes) 000187[1] 1780808209.089520 mapread 0x086da2 thru 0x0878d8 (0xb37 bytes) 000188[1] 1780808209.106631 write 0x6cc1e4 thru 0x6cc8e4 (0x701 bytes) 000189[1] 1780808209.132816 mapwrite 0x61e634 thru 0x61ec5b (0x628 bytes) 000191[0] 1780808209.177779 trunc from 0x7d0372 to 0x415375 (0x3baffc bytes) 000192[0] 1780808209.194724 write 0x1aaba6 thru 0x1acb14 (0x1f6f bytes) 000194[0] 1780808209.227333 write 0x83f4e8 thru 0x840eaa (0x19c3 bytes) 000198[0] 1780808209.255584 read 0x295570 thru 0x296dfb (0x188c bytes) 000199[1] 1780808209.258686 write 0x8306e6 thru 0x830bf6 (0x511 bytes) 000200[0] 1780808209.274142 write 0x055589 thru 0x055f1a (0x992 bytes) 000202[1] 1780808209.282496 read 0x5ada65 thru 0x5af878 (0x1e14 bytes) 000203[0] 1780808209.294970 mapwrite 0x52c77c thru 0x52ce3f (0x6c4 bytes) 000205[0] 1780808209.316014 mapwrite 0x667e53 thru 0x6687ac (0x95a bytes) 000207[0] 1780808209.337832 mapwrite 0x6ab25a thru 0x6abf62 (0xd09 bytes) 000209[0] 1780808209.361393 mapread 0x381ae1 thru 0x3831b7 (0x16d7 bytes) 000211[1] 1780808209.376950 mapread 0x760343 thru 0x761392 (0x1050 bytes) 000212[0] 1780808209.391663 mapread 0x776b7c thru 0x777712 (0xb97 bytes) 000217[0] 1780808209.415124 write 0x156977 thru 0x157b41 (0x11cb bytes) 000218[1] 1780808209.419039 read 0x1768d2 thru 0x177c8f (0x13be bytes) 000220[0] 1780808209.427289 mapread 0x31d685 thru 0x31e711 (0x108d bytes) 000221[1] 1780808209.430953 mapread 0x12c745 thru 0x12d85c (0x1118 bytes) 000224[0] 1780808209.435579 trunc from 0x840eab to 0x648594 (0x1f8916 bytes) 000227[0] 1780808209.455048 write 0x4aa541 thru 0x4aa890 (0x350 bytes) 000229[0] 1780808209.462095 read 0x2aa5a3 thru 0x2abbda (0x1638 bytes) 000230[0] 1780808209.470112 write 0x826665 thru 0x82702b (0x9c7 bytes) 000232[0] 1780808209.483087 mapwrite 0x308b6f thru 0x309b5e (0xff0 bytes) 000233[1] 1780808209.504295 mapread 0x12d134 thru 0x12e3a9 (0x1276 bytes) 000235[0] 1780808209.518997 read 0x2fb0fd thru 0x2fbe37 (0xd3b bytes) 000238[1] 1780808209.536592 write 0x872691 thru 0x872c60 (0x5d0 bytes) 000241[0] 1780808209.563416 mapread 0x419aa1 thru 0x41a365 (0x8c5 bytes) 000243[1] 1780808209.580019 trunc from 0x872c61 to 0x5bb725 (0x2b753b bytes) 000245[1] 1780808209.599567 write 0x1eecb8 thru 0x1eeddb (0x124 bytes) 000247[1] 1780808209.609457 mapread 0x2d5b89 thru 0x2d72f0 (0x1768 bytes) 000250[0] 1780808209.629569 read 0x1778ba thru 0x178e51 (0x1598 bytes) 000251[1] 1780808209.639712 mapread 0x33eb06 thru 0x33f4d9 (0x9d4 bytes) 000252[0] 1780808209.655214 mapread 0x4d2491 thru 0x4d25ce (0x13e bytes) 000253[0] 1780808209.669467 write 0x9a0377 thru 0x9a06cd (0x357 bytes) 000254[1] 1780808209.680765 trunc from 0x9a06ce to 0x1bdd52 (0x7e297b bytes) 000258[0] 1780808209.765449 mapread 0x192a69 thru 0x1935fc (0xb94 bytes) 000261[0] 1780808209.789916 read 0x120904 thru 0x121303 (0xa00 bytes) 000262[1] 1780808209.810555 mapwrite 0x51d343 thru 0x51e4ca (0x1188 bytes) 000265[0] 1780808209.880780 trunc from 0x51e4cb to 0x937e93 (0x4199c9 bytes) 000266[0] 1780808209.916494 mapwrite 0x76e7b2 thru 0x770578 (0x1dc7 bytes) 000267[0] 1780808209.994410 trunc from 0x937e94 to 0x1b29dd (0x7854b6 bytes) 000269[1] 1780808210.026133 trunc from 0x1b29de to 0x61ee5e (0x46c481 bytes) 000270[0] 1780808210.047713 mapwrite 0x65cf16 thru 0x65deb5 (0xfa0 bytes) 000272[1] 1780808210.106531 read 0x004ac9 thru 0x0051e5 (0x71d bytes) 000273[0] 1780808210.123904 mapwrite 0x796f76 thru 0x798b41 (0x1bcc bytes) 000274[0] 1780808210.172374 mapwrite 0x19b5df thru 0x19ceb6 (0x18d8 bytes) 000279[0] 1780808210.216906 read 0x6a8b2b thru 0x6aa7fa (0x1cd0 bytes) 000282[1] 1780808210.225675 trunc from 0x798b42 to 0x234ae4 (0x56405d bytes) 000284[1] 1780808210.239368 mapread 0x0c1866 thru 0x0c2b3d (0x12d8 bytes) 000285[0] 1780808210.243583 read 0x092ed2 thru 0x093d2a (0xe59 bytes) 000288[1] 1780808210.263753 trunc from 0x234ae5 to 0x53f38a (0x30a8a6 bytes) 000290[0] 1780808210.279753 mapwrite 0x653af2 thru 0x654b87 (0x1096 bytes) 000291[1] 1780808210.326494 trunc from 0x654b88 to 0x88eabf (0x239f38 bytes) 000294[1] 1780808210.355876 trunc from 0x88eac0 to 0x938bfe (0xaa13f bytes) 000295[0] 1780808210.364899 read 0x7b0d59 thru 0x7b0df1 (0x99 bytes) 000298[0] 1780808210.378278 mapread 0x928a9f thru 0x92a2d4 (0x1836 bytes) 000301[0] 1780808210.391020 read 0x0e111c thru 0x0e263e (0x1523 bytes) 000304[1] 1780808210.394147 mapread 0x7565fa thru 0x757432 (0xe39 bytes) 000307[1] 1780808210.399430 mapread 0x63f8c2 thru 0x640db4 (0x14f3 bytes) 000311[1] 1780808210.404963 mapread 0x7da8fe thru 0x7db544 (0xc47 bytes) 000312[0] 1780808210.409524 mapread 0x505795 thru 0x506ea5 (0x1711 bytes) 000313[0] 1780808210.413060 mapread 0x0c57e1 thru 0x0c6f17 (0x1737 bytes) 000318[0] 1780808210.418569 trunc from 0x938bff to 0x918efb (0x1fd03 bytes) 000319[0] 1780808210.433990 mapwrite 0x6d942a thru 0x6daa3d (0x1614 bytes) 000322[1] 1780808210.459531 read 0x0d6b21 thru 0x0d857e (0x1a5e bytes) 000323[1] 1780808210.469354 mapread 0x5bf310 thru 0x5bfbed (0x8de bytes) 000325[1] 1780808210.486668 read 0x55cf00 thru 0x55de01 (0xf02 bytes) 000326[0] 1780808210.489451 read 0x8b36ab thru 0x8b5454 (0x1daa bytes) 000327[1] 1780808210.493692 trunc from 0x918efc to 0x98a93c (0x71a41 bytes) 000328[1] 1780808210.514005 write 0x327ce4 thru 0x329580 (0x189d bytes) 000331[0] 1780808210.521460 write 0x934671 thru 0x9362ae (0x1c3e bytes) 000334[1] 1780808210.558201 write 0x72bd81 thru 0x72c8b8 (0xb38 bytes) 000335[1] 1780808210.568129 mapwrite 0x47cb25 thru 0x47dced (0x11c9 bytes) 000341[1] 1780808210.641631 mapread 0x4d4e5b thru 0x4d5fc8 (0x116e bytes) 000344[1] 1780808210.671083 mapread 0x1beccd thru 0x1c0815 (0x1b49 bytes) 000345[1] 1780808210.705239 read 0x923549 thru 0x9247c9 (0x1281 bytes) 000346[0] 1780808210.707899 mapwrite 0x93b251 thru 0x93b570 (0x320 bytes) 000348[0] 1780808210.744958 mapread 0x55e44a thru 0x55f7bb (0x1372 bytes) 000349[1] 1780808210.759622 mapread 0x48542d thru 0x486b35 (0x1709 bytes) 000350[0] 1780808210.765717 mapread 0x33460b thru 0x3348a0 (0x296 bytes) 000352[0] 1780808210.768399 write 0x6ee6c0 thru 0x6ef2c8 (0xc09 bytes) 000354[1] 1780808210.786791 read 0x91c99d thru 0x91d473 (0xad7 bytes) 000357[1] 1780808210.800534 read 0x855113 thru 0x856a33 (0x1921 bytes) 000358[1] 1780808210.806982 write 0x439893 thru 0x43b7a8 (0x1f16 bytes) 000359[0] 1780808210.817913 mapread 0x345173 thru 0x345ded (0xc7b bytes) 000363[1] 1780808210.833460 trunc from 0x98a93d to 0x3c05d7 (0x5ca365 bytes) 000365[0] 1780808210.846979 read 0x0f742a thru 0x0f8094 (0xc6b bytes) 000367[1] 1780808210.859693 mapwrite 0x78fef7 thru 0x791cb2 (0x1dbc bytes) 000370[0] 1780808210.916904 write 0x2430a9 thru 0x243e69 (0xdc1 bytes) 000371[0] 1780808210.929532 read 0x33d578 thru 0x33d5a6 (0x2f bytes) 000372[0] 1780808210.940823 trunc from 0x791cb3 to 0x982635 (0x1f0983 bytes) 000373[1] 1780808210.963394 mapread 0x17c20e thru 0x17c5d9 (0x3cc bytes) 000376[1] 1780808210.986650 mapread 0x5adac4 thru 0x5ae3f4 (0x931 bytes) 000377[1] 1780808210.996227 trunc from 0x982636 to 0x6f28a7 (0x28fd8e bytes) 000378[0] 1780808211.007016 mapread 0x21f906 thru 0x221389 (0x1a84 bytes) 000381[1] 1780808211.027174 trunc from 0x6f28a8 to 0x1d5f3c (0x51c96b bytes) 000383[0] 1780808211.036628 trunc from 0x1d5f3d to 0x1cf6ea (0x6852 bytes) 000385[1] 1780808211.050005 mapwrite 0x315e7c thru 0x3164e5 (0x66a bytes) 000386[1] 1780808211.086511 write 0x0c4b5d thru 0x0c60ff (0x15a3 bytes) 000388[0] 1780808211.098047 read 0x2db14e thru 0x2db954 (0x807 bytes) 000391[0] 1780808211.132181 mapwrite 0x27eb2b thru 0x280a78 (0x1f4e bytes) 000392[0] 1780808211.176174 mapwrite 0x917004 thru 0x918495 (0x1492 bytes) 000396[0] 1780808211.299517 mapread 0x1586c0 thru 0x158c9e (0x5df bytes) 000397[0] 1780808211.306865 trunc from 0x918496 to 0x24c6da (0x6cbdbb bytes) 000398[1] 1780808211.330177 mapwrite 0x3b1c9c thru 0x3b3c6e (0x1fd3 bytes) 000401[0] 1780808211.395676 trunc from 0x3b3c6f to 0x76c629 (0x3b89bb bytes) 000402[0] 1780808211.407893 write 0x8aecdb thru 0x8b0759 (0x1a7f bytes) 000403[1] 1780808211.419359 write 0x3c3463 thru 0x3c4bbe (0x175c bytes) 000406[0] 1780808211.448792 mapwrite 0x0735e7 thru 0x073c1c (0x636 bytes) 000408[1] 1780808211.497785 write 0x9a44fa thru 0x9a4b02 (0x609 bytes) 000410[1] 1780808211.514525 mapread 0x39005f thru 0x390764 (0x706 bytes) 000414[0] 1780808211.542555 write 0x0f7bfa thru 0x0f9951 (0x1d58 bytes) 000415[1] 1780808211.551649 read 0x598c6d thru 0x599202 (0x596 bytes) 000416[0] 1780808211.562533 mapread 0x8d5ac9 thru 0x8d5ca2 (0x1da bytes) 000417[1] 1780808211.577795 mapread 0x1f08a7 thru 0x1f2649 (0x1da3 bytes) 000419[1] 1780808211.614553 mapwrite 0x5fbf4c thru 0x5fdb8d (0x1c42 bytes) 000421[0] 1780808211.673724 trunc from 0x9a4b03 to 0x4fe492 (0x4a6670 bytes) 000424[0] 1780808211.707800 mapread 0x35f825 thru 0x35ff40 (0x71c bytes) 000425[1] 1780808211.714684 mapwrite 0x82634a thru 0x827bf8 (0x18af bytes) 000426[1] 1780808211.743466 trunc from 0x827bf9 to 0x104c7c (0x722f7c bytes) 000428[1] 1780808211.753743 mapread 0x0cc1a6 thru 0x0ccade (0x939 bytes) 000431[1] 1780808211.765797 read 0x00880c thru 0x009473 (0xc68 bytes) 000435[0] 1780808211.791066 write 0x7126a3 thru 0x7133e3 (0xd41 bytes) 000438[0] 1780808211.817612 mapread 0x23c102 thru 0x23d01e (0xf1d bytes) 000443[1] 1780808211.845236 write 0x409e3f thru 0x40a637 (0x7f9 bytes) 000444[0] 1780808211.856869 write 0x1476ad thru 0x1492a8 (0x1bfc bytes) 000445[1] 1780808211.864703 write 0x12dd43 thru 0x12e348 (0x606 bytes) 000446[1] 1780808211.874372 write 0x2dfd56 thru 0x2e0ee8 (0x1193 bytes) 000449[1] 1780808211.895667 write 0x19f3ba thru 0x1a0dd9 (0x1a20 bytes) 000450[0] 1780808211.910728 mapwrite 0x307711 thru 0x3086aa (0xf9a bytes) 000451[0] 1780808211.959273 mapread 0x63d323 thru 0x63ee09 (0x1ae7 bytes) 000454[1] 1780808211.986898 mapwrite 0x79d548 thru 0x79ef4b (0x1a04 bytes) 000455[1] 1780808212.073462 trunc from 0x79ef4c to 0x58c1d7 (0x212d74 bytes) 000456[0] 1780808212.083647 trunc from 0x58c1d8 to 0x4cdd93 (0xbe444 bytes) 000460[0] 1780808212.107350 trunc from 0x4cdd94 to 0x761e54 (0x2940c1 bytes) 000463[0] 1780808212.129854 trunc from 0x761e55 to 0x36d066 (0x3f4dee bytes) 000465[1] 1780808212.144472 mapwrite 0x98240b thru 0x982469 (0x5f bytes) 000466[1] 1780808212.211672 trunc from 0x98246a to 0x8ae405 (0xd4064 bytes) 000467[0] 1780808212.220007 write 0x30f49b thru 0x310809 (0x136f bytes) 000468[0] 1780808212.249226 mapread 0x47430f thru 0x475a3b (0x172d bytes) 000473[0] 1780808212.274029 read 0x0bd921 thru 0x0be1b6 (0x896 bytes) 000477[1] 1780808212.285701 write 0x98fc3e thru 0x990b6e (0xf31 bytes) 000478[1] 1780808212.293848 mapwrite 0x927f6f thru 0x929275 (0x1307 bytes) 000480[0] 1780808212.322646 read 0x3baf03 thru 0x3bc820 (0x191e bytes) 000481[1] 1780808212.329856 trunc from 0x990b6f to 0x2335b4 (0x75d5ba bytes) 000484[0] 1780808212.352999 read 0x08f878 thru 0x090bd5 (0x135e bytes) 000485[1] 1780808212.363564 mapread 0x137055 thru 0x1375bb (0x567 bytes) 000488[0] 1780808212.371641 mapwrite 0x20b547 thru 0x20d270 (0x1d2a bytes) 000489[0] 1780808212.399747 mapread 0x0ad6d3 thru 0x0adb3a (0x468 bytes) 000491[1] 1780808212.416313 read 0x0b8200 thru 0x0b946f (0x1270 bytes) 000492[0] 1780808212.444176 read 0x0fd509 thru 0x0fe004 (0xafc bytes) 000493[1] 1780808212.466284 mapread 0x0f557a thru 0x0f6723 (0x11aa bytes) 000494[1] 1780808212.467128 mapwrite 0x4fb482 thru 0x4fccac (0x182b bytes) 000495[0] 1780808212.537197 write 0x1b4682 thru 0x1b5a40 (0x13bf bytes) 000496[0] 1780808212.572673 trunc from 0x4fccad to 0x0372b6 (0x4c59f6 bytes) 000497[1] 1780808212.584669 mapread 0x02ce1b thru 0x02e6d5 (0x18bb bytes) 000498[1] 1780808212.594117 read 0x0324ff thru 0x033e8a (0x198c bytes) 000499[0] 1780808212.597950 mapread 0x01d8d1 thru 0x01e9d3 (0x1103 bytes) All operations completed A-OK! PASS 16b (10s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 00:56:54 (1780808214) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 00:56:55 (1780808215) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.138247 s, 60.7 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.110295 s, 76.1 MB/s ldlm.namespaces.MGC192.168.202.111@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8ab686465000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8ab687ae7800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8ab687ae7800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8ab687ae7800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.36884 s, 31.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.22498 s, 85.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.01117 s, 104 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.38976 s, 43.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.40165 s, 43.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.42702 s, 73.5 MB/s PASS 16d (17s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 00:57:12 (1780808232) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.104826 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510108 s, 20.6 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 00:57:15 (1780808235) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 00:57:38 (1780808258) TIMEOUT PASS 16g (23s) == sanityn test 16h: mmap read after truncate file ======= 00:58:01 (1780808281) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (3s) == sanityn test 16i: read after truncate file ============ 00:58:04 (1780808284) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (3s) == sanityn test 16j: race dio with buffered i/o ========== 00:58:07 (1780808287) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.532568 s, 49.2 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.629307 s, 41.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.16584 s, 22.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.734978 s, 35.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.934767 s, 28.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.810992 s, 32.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.04087 s, 25.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.670607 s, 39.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.810595 s, 32.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.17044 s, 22.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.20004 s, 21.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.849996 s, 30.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.948717 s, 27.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.13798 s, 23.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.22385 s, 21.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.15733 s, 22.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.25589 s, 20.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.37364 s, 19.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.41067 s, 18.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (15s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 00:58:22 (1780808302) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Sun Jun 7 00:58:38 EDT 2026 + fsx 2 done: rc=0 Sun Jun 7 00:58:38 EDT 2026 + fsx 3 done: rc=0 Sun Jun 7 00:58:38 EDT 2026 + fsx 4 done: rc=0 Sun Jun 7 00:58:38 EDT 2026 + fsx 5 done: rc=0 Sun Jun 7 00:58:38 EDT 2026 PASS 16k (20s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 00:58:42 (1780808322) fail_loc=0x8000030a PASS 17 (5s) == sanityn test 18: mmap sanity check =========================================================================================== 00:58:47 (1780808327) mmap test1: basic mmap operation (PASS, 0.031699s) mmap test2: MAP_PRIVATE not write back (PASS, 0.018052s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.7701s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.2146s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.081297s) mmap test6: check mmap write/read content on two nodes (PASS, 0.12045s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.54824s) mmap test11: multi-thread shared mmap (PASS, 0.56925s) mmap test10: mtime not change for readonly mmap access (PASS, 1.055s) PASS 18 (14s) == sanityn test 19: test concurrent uncached read races ========================================================================= 00:59:01 (1780808341) oleg211-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg211-client: oleg211-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 00:59:03 (1780808343) PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 00:59:06 (1780808346) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (3s) == sanityn test 23: others should see updated atime while another read============================================================== 00:59:09 (1780808349) now is 1780808350 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 new atime is 1780808411 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 01:00:13 (1780808413) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2048 2206592 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2048 2206592 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.9K 386 524.5K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 403 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.9K 386 524.5K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 403 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 2.0M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 32.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537447 386 537061 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118067 403 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117232 400 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2048 2206592 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.9K 386 524.5K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 403 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537447 386 537061 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118067 403 117664 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 117232 400 116832 1% /mnt/lustre[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537447 386 537061 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118067 403 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117232 400 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2048 2206592 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2048 2206592 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 01:00:16 (1780808416) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 01:00:19 (1780808419) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 01:00:22 (1780808422) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 01:00:23 (1780808423) utime: good mknod times 1780808422 <= 1780808423 <= 1780808423 for /mnt/lustre/f26a utime: good mknod times 1780808422 <= 1780808423 <= 1780808423 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 01:00:27 (1780808427) PASS 26b (5s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 01:00:31 (1780808431) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-06-07 01:00:32.000000000 -0400 Birth: 2026-06-07 01:00:32.000000000 -0400 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-06-07 01:00:32.000000000 -0400 Birth: 2026-06-07 01:00:32.000000000 -0400 PASS 26c (4s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 01:00:35 (1780808435) 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 01:00:40 (1780808440) fail_loc=0x314 PASS 31a (4s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 01:00:44 (1780808444) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg211-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.111@tcp:/lustre 7542784 39936 7498752 1% /mnt/lustre2 PASS 31b (14s) == sanityn test 31r: open-rename(replace) race =========== 01:00:58 (1780808458) -rw-r--r-- 1 root root 0 Jun 7 01:00 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (6s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 01:01:04 (1780808464) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 42990 Terminated tail -f $tfile PASS 31s (4s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 01:01:08 (1780808468) PASS 31t (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 01:01:12 (1780808472) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 01:01:14 (1780808474) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 01:01:15 (1780808475) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 01:01:17 (1780808477) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 01:01:18 (1780808478) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ab686465000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8ab686465000.ost_server_uuid in FULL state after 0 sec oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8ab686465000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8ab686465000.ost_server_uuid in IDLE state after 0 sec oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ab686465000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8ab686465000.ost_server_uuid in FULL state after 0 sec oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8ab686465000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8ab686465000.ost_server_uuid in IDLE state after 0 sec oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ab686465000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8ab686465000.ost_server_uuid in IDLE state after 0 sec oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8ab686465000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8ab686465000.ost_server_uuid in IDLE state after 0 sec PASS 34 (92s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 01:02:51 (1780808571) fail_loc=0x80000317 Race attempt 0 Wait for 49029 49163 for 60 sec... - open/close 3520 (time 1780808583.34 total 10.00 last 351.97) total: 4000 open/close in 11.45 seconds: 349.45 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.111@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre 192.168.202.111@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 01:03:55 (1780808635) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg211-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.348294 s, 151 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.3962 s, 132 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.474948 s, 110 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.515847 s, 102 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.440394 s, 119 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7498752) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.462774 s, 113 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7494656) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.480507 s, 109 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.430164 s, 122 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.44501 s, 118 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.427185 s, 123 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7498752) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.414813 s, 126 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7498752) after_dd(7445504) after(7496704) PASS 36 (232s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 01:07:47 (1780808867) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 - create 8990 (time 1780808878.54 total 10.00 last 898.96) total: 10000 create in 11.01 seconds: 908.13 ops/second PASS 37 (14s) == sanityn test 39a: file mtime does not change after rename ========================================================== 01:08:01 (1780808881) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 01:08:03 (1780808883) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 01:08:06 (1780808886) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 01:08:09 (1780808889) fail_loc=0x411 fail_loc=0 PASS 39d (2s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 01:08:12 (1780808892) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 01:08:20 (1780808900) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 01:08:27 (1780808907) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 01:08:35 (1780808915) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 01:08:43 (1780808923) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 01:08:48 (1780808928) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 01:08:53 (1780808933) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 01:08:58 (1780808938) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 01:09:03 (1780808943) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 01:09:08 (1780808948) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 01:09:13 (1780808953) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 01:09:18 (1780808958) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 2318: lctl set_param -n fail_loc=0 2>/dev/null || true: No such file or directory Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 01:09:23 (1780808963) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (712s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 01:21:15 (1780809675) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 01:21:20 (1780809680) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 01:21:24 (1780809684) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 01:21:29 (1780809689) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 01:21:34 (1780809694) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 01:21:39 (1780809699) Conflict PASS 42f (6s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 01:21:45 (1780809705) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 01:21:50 (1780809710) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 01:21:55 (1780809715) PASS 43a (16s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 01:22:11 (1780809731) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 01:22:16 (1780809736) Conflict PASS 43c (5s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 01:22:21 (1780809741) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 01:22:26 (1780809746) Conflict PASS 43e (6s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 01:22:32 (1780809752) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 01:22:37 (1780809757) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 01:22:41 (1780809761) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 01:22:46 (1780809766) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 01:22:47 (1780809767) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (38s) == sanityn test 43k: unlink vs create ==================== 01:23:25 (1780809805) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (425s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 01:30:30 (1780810230) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Sun Jun 7 01:30:33 EDT 2026 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 01:30:35 (1780810235) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 01:30:40 (1780810240) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 01:30:46 (1780810246) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 01:30:51 (1780810251) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 01:30:56 (1780810256) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 01:31:01 (1780810261) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 01:31:06 (1780810266) Conflict PASS 44h (6s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 01:31:12 (1780810272) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 01:31:13 (1780810273) PASS 45a (27s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 01:31:40 (1780810300) Conflict PASS 45b (6s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 01:31:46 (1780810306) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 01:31:51 (1780810311) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 01:31:56 (1780810316) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 01:32:01 (1780810321) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 01:32:06 (1780810326) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 01:32:12 (1780810332) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 01:32:16 (1780810336) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 01:32:17 (1780810337) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (419s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 01:39:16 (1780810756) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 01:39:21 (1780810761) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 01:39:26 (1780810766) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 01:39:31 (1780810771) Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 01:39:36 (1780810776) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 01:39:41 (1780810781) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 01:39:46 (1780810786) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 01:39:51 (1780810791) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 01:39:56 (1780810796) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 01:39:57 (1780810797) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 01:39:58 (1780810798) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 01:39:59 (1780810799) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 01:40:00 (1780810800) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 01:40:01 (1780810801) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 01:40:02 (1780810802) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 01:40:04 (1780810804) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 01:40:05 (1780810805) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00487278 s, 2.1 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 01:40:12 (1780810812) 0+1 records in 0+1 records out 158 bytes copied, 0.00357838 s, 44.2 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3581: kill: (217447) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 01:40:16 (1780810816) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0028266 s, 362 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151093 s, 69.4 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 01:40:31 (1780810831) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00388271 s, 264 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 01:40:37 (1780810837) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147753 s, 71.0 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 01:40:40 (1780810840) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 01:40:45 (1780810845) llite.lustre-ffff8ab686465000.xattr_cache=1 llite.lustre-ffff8ab687ae7800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 01:40:45.000000000 -0400 Modify: 2026-06-07 01:40:45.000000000 -0400 Change: 2026-06-07 01:40:45.000000000 -0400 Birth: 2026-06-07 01:40:46.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505014 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 01:40:51.000000000 -0400 Modify: 2026-06-07 01:40:51.000000000 -0400 Change: 2026-06-07 01:40:51.000000000 -0400 Birth: 2026-06-07 01:40:51.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 01:40:56.000000000 -0400 Modify: 2026-06-07 01:40:56.000000000 -0400 Change: 2026-06-07 01:40:56.000000000 -0400 Birth: 2026-06-07 01:40:57.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505016 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 01:40:56.000000000 -0400 Modify: 2026-06-07 01:40:56.000000000 -0400 Change: 2026-06-07 01:40:56.000000000 -0400 Birth: 2026-06-07 01:40:57.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505019 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 01:41:02.000000000 -0400 Modify: 2026-06-07 01:41:02.000000000 -0400 Change: 2026-06-07 01:41:02.000000000 -0400 Birth: 2026-06-07 01:41:02.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 01:41:02.000000000 -0400 Modify: 2026-06-07 01:41:02.000000000 -0400 Change: 2026-06-07 01:41:02.000000000 -0400 Birth: 2026-06-07 01:41:02.000000000 -0400 PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 01:41:09 (1780810869) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 01:41:16 (1780810876) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 01:41:24 (1780810884) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 01:41:37 (1780810897) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 01:41:47 (1780810907) SKIP: sanityn test_55e needs >= 2 MDTs SKIP 55e (1s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 01:41:48 (1780810908) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (14s) == sanityn test 55g: rename: race with trylock =========== 01:42:02 (1780810922) mdt.lustre-MDT0000.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre2/d55g.sanityn/b1/b2' to a subdirectory of itself, '/mnt/lustre2/d55g.sanityn/a1/a2/a3/a4/b2' We survived race with trylock test fail_loc=0x0 PASS 55g (14s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 01:42:17 (1780810937) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1780810937 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 1.84967 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (35s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 01:42:52 (1780810972) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1780810974 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 3.13011 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (45s) == sanityn test 60: Verify data_version behaviour ======== 01:43:37 (1780811017) PASS 60 (3s) == sanityn test 70a: cd directory && rm directory ======== 01:43:40 (1780811020) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 01:43:43 (1780811023) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 01:43:46 (1780811026) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00455791 s, 14.4 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 01:43:47 (1780811027) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0071966 s, 9.1 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 01:43:48 (1780811028) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 01:43:50 (1780811030) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00446158 s, 14.7 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 01:43:51 (1780811031) llite.lustre-ffff8ab686465000.xattr_cache=1 llite.lustre-ffff8ab687ae7800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 01:43:53 (1780811033) llite.lustre-ffff8ab686465000.xattr_cache=1 llite.lustre-ffff8ab687ae7800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 01:43:56 (1780811036) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00304694 s, 336 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00298948 s, 343 kB/s 230211: get lock2 230213: get lock1 230211: done 230211: get lock1 230213: done 230213: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 230213: cannot get lock2: Resource deadlock avoided 230213: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 230211: cannot get lock1: Resource deadlock avoided 230211: done 230211: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 01:44:03 (1780811043) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0251584 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117579 s, 89.2 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4264: 230803 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 01:44:10 (1780811050) opening files: ........ closing files: ........ PASS 76 (23s) == sanityn test 77a: check FIFO NRS policy =============== 01:44:33 (1780811073) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.133279 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199317 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475609 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989081 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106269 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110846 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140921 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120001 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149041 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126798 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15536 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16395 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139161 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157756 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154503 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168842 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154113 s, 6.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086172 s, 122 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171008 s, 61.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125655 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069279 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00464352 s, 226 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021112 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229958 s, 45.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00500546 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114536 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407577 s, 25.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980593 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618557 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228735 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122872 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745113 s, 141 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072303 s, 145 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 01:44:36 (1780811076) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.166298 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863553 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098296 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109771 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135492 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13194 s, 7.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133716 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127976 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141079 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13955 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138872 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150996 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144435 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147342 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191684 s, 5.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151479 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150382 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015384 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224494 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200274 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239233 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252712 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169099 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281987 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955079 s, 110 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133769 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037568 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172018 s, 61.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112403 s, 93.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147807 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125664 s, 83.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211654 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927763 s, 113 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.130586 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539784 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508136 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603973 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541377 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0886884 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706606 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0900615 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800624 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099066 s, 10.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140673 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172749 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156838 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178799 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162591 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176356 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187987 s, 5.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192949 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807494 s, 130 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011962 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191336 s, 54.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557826 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00509233 s, 206 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120057 s, 87.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210308 s, 49.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187238 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121669 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135054 s, 77.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239217 s, 43.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136258 s, 77.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00440591 s, 238 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00470858 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912009 s, 115 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 01:44:40 (1780811080) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.184791 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428086 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361189 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596654 s, 17.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586221 s, 17.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510409 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408525 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477924 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500507 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623099 s, 16.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655701 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0984595 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114206 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109594 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11827 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116958 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134241 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133986 s, 78.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153101 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0039039 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276348 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00304246 s, 345 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105508 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00249379 s, 420 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195371 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283212 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00366192 s, 286 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00369013 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309354 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326294 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166958 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376415 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261938 s, 40.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.158756 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382745 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454898 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435751 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552888 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785666 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100145 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083517 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861765 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801312 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875101 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09532 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102747 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887715 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104358 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901658 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101308 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970416 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016776 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170507 s, 61.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217264 s, 48.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123148 s, 85.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212228 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254356 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211919 s, 49.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137736 s, 76.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170991 s, 61.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109774 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378744 s, 277 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643471 s, 163 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136603 s, 76.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307973 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227116 s, 46.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.213685 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222683 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738929 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815129 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829296 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918631 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809327 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843678 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819589 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791989 s, 13.2 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=3 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085036 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625026 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890148 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100402 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683738 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114495 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10462 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133719 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153787 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105939 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134223 s, 78.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00414883 s, 253 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142202 s, 73.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211703 s, 49.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240867 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142732 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226376 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859667 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00368808 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011516 s, 91.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566198 s, 185 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890881 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121445 s, 86.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 01:44:46 (1780811086) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.233112 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420005 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379818 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535207 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748069 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845979 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650423 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686232 s, 15.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927516 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852048 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109723 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875063 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983272 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118883 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102736 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111172 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105953 s, 9.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197286 s, 53.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145888 s, 71.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186838 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172561 s, 60.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232363 s, 45.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025311 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209408 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210259 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167244 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209456 s, 50.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00522275 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899998 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123531 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00529043 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255479 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115972 s, 90.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.234384 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350621 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422227 s, 24.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773427 s, 13.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723492 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899324 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870077 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687697 s, 15.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863676 s, 12.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11514 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110716 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993151 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965819 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862162 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980364 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123485 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124406 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165511 s, 63.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247506 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00443603 s, 236 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192182 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159778 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183528 s, 57.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176921 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016805 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217325 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222573 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275843 s, 38.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253995 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260878 s, 40.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280762 s, 37.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=3 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280558 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00437171 s, 240 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.19521 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199323 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410993 s, 25.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656178 s, 16.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860656 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983397 s, 10.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993537 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113797 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132957 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124274 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12241 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117687 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123959 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121655 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117008 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129516 s, 8.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135231 s, 77.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00452293 s, 232 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138883 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185305 s, 56.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181455 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773584 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00471649 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345446 s, 30.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231534 s, 45.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0046557 s, 225 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333614 s, 31.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177121 s, 59.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164571 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694867 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00485686 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00520409 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579132 s, 181 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (6s) == sanityn test 77e: check TBF NID nrs policy ============ 01:44:52 (1780811092) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.11@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.258274 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339162 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575346 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569489 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604022 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0567173 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919637 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825605 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969054 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821009 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864121 s, 12.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796387 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925097 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769608 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111912 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880308 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843382 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206711 s, 50.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229177 s, 45.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167224 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185783 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00450013 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177505 s, 59.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025684 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172428 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352328 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240672 s, 43.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314042 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289401 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130596 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00372183 s, 282 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00405331 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00335433 s, 313 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.170283 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267961 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0542778 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465202 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659865 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458239 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0696483 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062772 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538775 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913606 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652319 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873603 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11165 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960006 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096104 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106813 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11866 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211173 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222213 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235487 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227466 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195715 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281292 s, 37.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104915 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164173 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240932 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715812 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104822 s, 100 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301169 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275274 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696618 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0052281 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670211 s, 156 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.167108 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340245 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263826 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732865 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786218 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488797 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966654 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828305 s, 12.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851132 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106682 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907082 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100726 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107924 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873356 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753798 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837591 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123198 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146703 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01216 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166188 s, 63.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106431 s, 98.5 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244403 s, 42.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916971 s, 114 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303754 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628853 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00495469 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451224 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168025 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00383057 s, 274 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347081 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209374 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342441 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130163 s, 80.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.20435 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292962 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496827 s, 21.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572058 s, 18.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615491 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509467 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0492441 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736846 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615654 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665291 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853193 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875641 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123511 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272553 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146356 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246674 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138772 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147094 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209343 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238389 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024545 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136765 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202984 s, 51.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360975 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284875 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362526 s, 28.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303519 s, 34.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=9 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435103 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192 s, 54.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374705 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392696 s, 26.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529681 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264011 s, 39.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (10s) == sanityn test 77f: check TBF JobID nrs policy ========== 01:45:02 (1780811102) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.281653 s, 59.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038829 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419988 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752246 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104491 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129445 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126001 s, 8.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149653 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140969 s, 7.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161871 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153691 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133361 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145381 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162405 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167149 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17608 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172257 s, 6.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266316 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169803 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313369 s, 33.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131084 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254684 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313206 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195489 s, 53.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178541 s, 58.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220721 s, 47.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200237 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186401 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174354 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657397 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347007 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297067 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104773 s, 100 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.131586 s, 127 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367397 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316196 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112075 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13078 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121241 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111409 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133819 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158036 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143411 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124995 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134625 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152534 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132819 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168659 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168534 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174418 s, 6.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014943 s, 70.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181538 s, 57.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206522 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180271 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205745 s, 51.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082629 s, 127 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209904 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155095 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176728 s, 59.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133749 s, 78.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342941 s, 30.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182907 s, 57.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191842 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140881 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143554 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383499 s, 27.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.163723 s, 102 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07454 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988562 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126494 s, 8.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147955 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175875 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115564 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183883 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174348 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025606 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175887 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192722 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170788 s, 61.4 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209072 s, 50.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182977 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208825 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482846 s, 21.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188052 s, 5.6 MB/sldlm.namespaces.lustre-OST0000-osc-ffff8ab686465000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288626 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219445 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0053843 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233174 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198057 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283219 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157082 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272043 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132805 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021488 s, 48.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385266 s, 27.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154872 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298366 s, 35.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147995 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197696 s, 5.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.161709 s, 104 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402526 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586873 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0827869 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07377 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771371 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108847 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077119 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101038 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109762 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100594 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948756 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119097 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924652 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126971 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101167 s, 10.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8ab686465000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100822 s, 10.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141034 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163793 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173513 s, 60.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158845 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150352 s, 69.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00382298 s, 274 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205682 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216707 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590293 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611635 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737397 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195145 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023512 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103208 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792824 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113432 s, 92.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (8s) == sanityn test 77g: Change TBF type directly ============ 01:45:10 (1780811110) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (5s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 01:45:15 (1780811115) oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg211-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg211-client: oleg211-server: ssh exited with exit code 2 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg211-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg211-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg211-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 01:45:19 (1780811119) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 01:45:28 (1780811128) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.135149 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217281 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455029 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488895 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583112 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0633491 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238458 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736294 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0749422 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083496 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101969 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943227 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851701 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934492 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091371 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966979 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104312 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948901 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321115 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298493 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289588 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275152 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261157 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312731 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247161 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335232 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361451 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566827 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00540545 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338304 s, 31.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00498068 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0443945 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00264395 s, 397 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95778 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4443 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (48s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 01:46:16 (1780811176) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23425 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484398 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681752 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090153 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878608 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078225 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097326 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961015 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0824476 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139801 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140061 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201644 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160386 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189634 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192831 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199838 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208382 s, 5.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266948 s, 39.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255081 s, 41.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214426 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372565 s, 28.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374065 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307897 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195484 s, 53.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179459 s, 58.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291838 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00538449 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380563 s, 27.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267394 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175125 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712196 s, 147 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217376 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195295 s, 53.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6983 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9678 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.22386 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771484 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735671 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751424 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795413 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735919 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933767 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883413 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100262 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154472 s, 6.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154437 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158948 s, 6.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139328 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168546 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131278 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173636 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148152 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178716 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189578 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020279 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166056 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167421 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00513399 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223684 s, 46.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134587 s, 77.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216735 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299442 s, 35.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231254 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00520624 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00386571 s, 271 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00426304 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00421515 s, 249 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00234763 s, 447 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8467 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9083 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (136s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 01:48:32 (1780811312) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23054 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833333 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813187 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0950249 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084198 s, 12.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103296 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860217 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881141 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961307 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158269 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158198 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141949 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174069 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162967 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157161 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159999 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17047 s, 6.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146386 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150506 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172645 s, 60.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193123 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214902 s, 48.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180305 s, 58.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224142 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128569 s, 81.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00403774 s, 260 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225728 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235009 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232837 s, 45.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214859 s, 48.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227504 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777201 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00188364 s, 557 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8309 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9498 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg211-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23863 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115462 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116687 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144625 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13885 s, 7.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147611 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138133 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130821 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14785 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141719 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157251 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137612 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137707 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153829 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15643 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14218 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140208 s, 7.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177842 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203994 s, 51.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228206 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238638 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250347 s, 41.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284317 s, 36.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303546 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299985 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735384 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143676 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160641 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279787 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359608 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337821 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198358 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153881 s, 68.1 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8433 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9295 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (139s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 01:50:51 (1780811451) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.338818 s, 49.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043932 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706268 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119369 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117864 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135861 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170917 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170547 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162601 s, 6.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164243 s, 6.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18219 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170934 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165211 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185678 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198898 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168203 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163894 s, 6.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210003 s, 49.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230338 s, 45.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258018 s, 40.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367398 s, 28.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224413 s, 46.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359256 s, 29.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018664 s, 56.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308138 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264746 s, 39.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207224 s, 50.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406461 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167034 s, 62.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157031 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126796 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217689 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839055 s, 125 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9495 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74441 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.11@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.11@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.247676 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328599 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788367 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669897 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077964 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114878 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112057 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1059 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145575 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130587 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153976 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136743 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162036 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230542 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248643 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288935 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273157 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355329 s, 29.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381486 s, 27.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489726 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368468 s, 28.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295878 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484464 s, 21.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329223 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356656 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310165 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369444 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348491 s, 30.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394819 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410693 s, 25.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537969 s, 19.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398275 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656389 s, 16.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93298 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74076 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.11@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.259416 s, 64.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393752 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10869 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133054 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134248 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175088 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173409 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10534 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182031 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193284 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176859 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178684 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163233 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173925 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184547 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185498 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173245 s, 6.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235267 s, 44.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305432 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028612 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342988 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159534 s, 65.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275121 s, 38.1 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033167 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012354 s, 84.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175104 s, 59.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116799 s, 89.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339388 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191092 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313017 s, 33.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276666 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135889 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198145 s, 52.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9849 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89097 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.442077 s, 38.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306169 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537539 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597806 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551849 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0491912 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456357 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571777 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456809 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546525 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983513 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794121 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110079 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288975 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302473 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324635 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358925 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177394 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178504 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019178 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180852 s, 58.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190802 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228501 s, 45.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176994 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729428 s, 144 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220557 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148152 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283567 s, 37.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969946 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025005 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228924 s, 45.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109915 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076203 s, 138 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.783 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72839 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91791 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73796 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.852986 s, 19.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286349 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0783914 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049656 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831144 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769366 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838468 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792012 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888212 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1331 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174156 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172451 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15885 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16568 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16022 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200093 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189437 s, 5.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104228 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170174 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135802 s, 77.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182851 s, 57.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177361 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143848 s, 72.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221031 s, 47.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004151 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175688 s, 59.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778659 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256334 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232275 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545584 s, 192 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00540042 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114907 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00452184 s, 232 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.69 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4384 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.239753 s, 70.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335976 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428257 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563128 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485436 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281595 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591556 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700569 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928585 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807937 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813247 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077902 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982951 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094589 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110246 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119018 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104277 s, 10.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117529 s, 89.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958372 s, 109 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215115 s, 48.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212333 s, 49.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159217 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232192 s, 45.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270851 s, 38.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163901 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111491 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294905 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274189 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314455 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575714 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00477404 s, 220 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0045226 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326342 s, 32.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91741 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72922 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.445548 s, 37.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306342 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422325 s, 24.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803964 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568872 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871168 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748411 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0953628 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121137 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101046 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120282 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112751 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104528 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228143 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247997 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331662 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339116 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014102 s, 74.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215253 s, 48.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200106 s, 52.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234241 s, 44.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199867 s, 52.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106986 s, 98.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200916 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176075 s, 59.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198871 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269998 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249937 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849093 s, 123 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00489154 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00425757 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004013 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013935 s, 75.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7832 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73254 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78369 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7332 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (359s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 01:56:50 (1780811810) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.313143 s, 53.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883772 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944545 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10134 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110829 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134699 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122591 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11919 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126635 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122828 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130935 s, 8.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120785 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112962 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137713 s, 7.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120439 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131853 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145499 s, 7.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133868 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242145 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208487 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318118 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023602 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232925 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269452 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850792 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248306 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339677 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031369 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290455 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374745 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168944 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575877 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118197 s, 88.7 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94972 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74148 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kb (43s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 01:57:33 (1780811853) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.11@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.11@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.268488 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186969 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663666 s, 15.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752597 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882298 s, 11.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700394 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836931 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938604 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100514 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965586 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0814769 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14852 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115874 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270681 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256749 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299625 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317048 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232468 s, 45.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204198 s, 51.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133207 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248587 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185945 s, 56.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260507 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904376 s, 116 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242341 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327239 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215021 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340702 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658578 s, 159 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00485268 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109676 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00338449 s, 310 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145856 s, 71.9 MB/s removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.99388 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74893 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kc (40s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 01:58:13 (1780811893) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.11@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.24002 s, 69.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540087 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774262 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128461 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14801 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150938 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147187 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146757 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144693 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184074 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143652 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155999 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16773 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171625 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184167 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201083 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191611 s, 5.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229956 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281333 s, 37.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279786 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235638 s, 44.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300235 s, 34.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280467 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169843 s, 61.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381365 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888173 s, 118 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213628 s, 49.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458582 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00354162 s, 296 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305518 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408593 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287887 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013374 s, 78.4 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9296 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96767 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kd (35s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 01:58:48 (1780811928) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.451756 s, 37.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327259 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261671 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308116 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056656 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049402 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616671 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059326 s, 17.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426 s, 24.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059911 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0780486 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079096 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112101 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.345585 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361391 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.401052 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409114 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206003 s, 50.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245057 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189958 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174478 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193094 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188185 s, 55.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209902 s, 50.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198972 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124173 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231112 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203753 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071378 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261382 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00400222 s, 262 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00391725 s, 268 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878009 s, 119 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.80623 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73297 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91914 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74556 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77ke (78s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 02:00:06 (1780812006) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.873706 s, 19.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608955 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781459 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0598889 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111362 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737079 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860423 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789142 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103686 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768844 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130812 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16917 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187604 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165368 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17991 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166724 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169971 s, 6.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468896 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231529 s, 45.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166209 s, 63.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144331 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199709 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191567 s, 54.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027571 s, 38.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239846 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510925 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325265 s, 32.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480733 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225872 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023694 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027403 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0602872 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411354 s, 25.5 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5181 s, 5.4 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9144 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kf (61s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 02:01:07 (1780812067) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.271941 s, 61.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229724 s, 45.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291501 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871574 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091789 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10584 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986595 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084675 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103025 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114949 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091309 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111787 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113448 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126331 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107071 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110654 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103057 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172028 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177843 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168868 s, 62.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238142 s, 44.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124039 s, 84.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207404 s, 50.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240607 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182334 s, 57.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026979 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224812 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214957 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00448359 s, 234 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00410813 s, 255 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00531534 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00509947 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274125 s, 38.3 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96622 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75152 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg211-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.455411 s, 36.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228369 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0519179 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707861 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117034 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112379 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121812 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158192 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1766 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17151 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168601 s, 6.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146931 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146986 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157319 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154934 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233254 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235803 s, 4.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206683 s, 50.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287626 s, 36.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259543 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260058 s, 40.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036901 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271296 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804369 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287364 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326186 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206034 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169147 s, 62.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159222 s, 65.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345345 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638521 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471671 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106541 s, 98.4 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79871 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74328 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.81138 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73825 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kg (123s) == sanityn test 77kh: Verify that Project ID support for NRS TBF rule ========================================================== 02:03:10 (1780812190) using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg211-client.virtnet /mnt/lustre (opts:) Stopping client oleg211-client.virtnet /mnt/lustre opts: Stopping clients: oleg211-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg211-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg211-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg211-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg211-server sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg211-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.11,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg211-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.11,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg211-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.11,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/64756/testresults/sanityn-zfs-rocky8.10_x86_64-rocky8.10_x86_64-retry1 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.11,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg211-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg211-server' oleg211-server: oleg211-server.virtnet: executing load_modules_local oleg211-server: Loading modules from /home/green/git/lustre-release/lustre oleg211-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg211-server: detected 4 online CPUs by sysfs oleg211-server: MODOPTS_LIBCFS= oleg211-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg211-server: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg211-client: oleg211-server: ssh exited with exit code 1 Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg211-server: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg211-client: oleg211-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg211-server: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg211-client: oleg211-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre Starting client oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre Started clients oleg211-client.virtnet: 192.168.202.111@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 Starting client oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 Started clients oleg211-client.virtnet: 192.168.202.111@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8ab68e174800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8ab68eb16800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ab68e174800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ab68eb16800.idle_timeout=debug disable quota as required ost.OSS.ost_io.nrs_policies=tbf projid ost.OSS.ost_io.nrs_tbf_rule=start ext_proj_rw projid={100} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.242412 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266993 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325408 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830883 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808618 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849325 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909865 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102653 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937329 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0876772 s, 12.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968035 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0895253 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109521 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227889 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258313 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293937 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308594 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162837 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147175 s, 71.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198873 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182807 s, 57.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264028 s, 39.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158864 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192111 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250876 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254023 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191987 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227396 s, 46.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260172 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263736 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280007 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252236 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272351 s, 38.5 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg211-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93731 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93522 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_proj_rw ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=tbf projid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w projid={100}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r projid={100}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.333353 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369175 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260173 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978677 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111376 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903415 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100418 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100651 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106756 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888045 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101512 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100491 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109362 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15294 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178559 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222627 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239281 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162533 s, 64.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138788 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177831 s, 59.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204032 s, 51.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279956 s, 37.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232251 s, 45.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104882 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586496 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175524 s, 59.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216956 s, 48.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257749 s, 40.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228991 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105278 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00462842 s, 227 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00383492 s, 273 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00368676 s, 284 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93354 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73874 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77kh (138s) == sanityn test 77ki: Add rule with unsupported TBF type should fail for generic TBF ========================================================== 02:05:28 (1780812328) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start ext_fail projid={100}&opcode={ost_write} rate=20: Invalid argument oleg211-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start ext_fail projid={100}&opcode={ost_write} rate=20': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77ki (10s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 02:05:38 (1780812338) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 02:05:42 (1780812342) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg211-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.84799 s, 272 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg211-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0115 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (52s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 02:06:34 (1780812394) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.148058 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218348 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566922 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608106 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603999 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709267 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448989 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709571 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05527 s, 19.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8ab68e174800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0810829 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101001 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106169 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121696 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108437 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101664 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114227 s, 9.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8ab68e174800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127721 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870414 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215949 s, 48.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155991 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208439 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175355 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195596 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228356 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224103 s, 46.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020231 s, 51.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269464 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00399173 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185656 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227556 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225472 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221952 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269475 s, 38.9 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93836 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97474 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.313513 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375933 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255059 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946033 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11236 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108 s, 9.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102593 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086926 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106991 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103119 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101877 s, 10.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11602 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105174 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171506 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168166 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237082 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208316 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129222 s, 81.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214344 s, 48.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157915 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214729 s, 48.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160996 s, 65.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164456 s, 63.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876544 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323318 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00333904 s, 314 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253427 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313069 s, 33.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250533 s, 41.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227567 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242388 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00233109 s, 450 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00240632 s, 436 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg211-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg211-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96256 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93535 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (65s) == sanityn test 77o: Changing rank should not panic ====== 02:07:39 (1780812459) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 02:07:43 (1780812463) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (42s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 02:08:25 (1780812505) mds.MDS.mdt.nrs_policies=tbf oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg211-client: oleg211-server: ssh exited with exit code 36 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg211-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg211-client: oleg211-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 02:08:38 (1780812518) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 272106 kill md_thread xargs PASS 77r (42s) == sanityn test 77s: Check TBF LRU shrinker ============== 02:09:20 (1780812560) jobid_name=%e.%p jobid_var=TEST_77s_JOBVAR mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start touch jobid={touch.*} rate=1000 mds.MDS.mdt.nrs_tbf_rule=start unlink jobid={unlink.*} rate=1000 touch jobid={touch.*} 1000, ref 10 touch jobid={touch.*} 1000, ref 0 unlink jobid={unlink.*} 1000, ref 10 unlink jobid={unlink.*} 1000, ref 0 mds.MDS.mdt.nrs_policies=fifo jobid_var=procname_uid jobid_name=%e.%u PASS 77s (17s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 02:09:37 (1780812577) oleg211-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg211-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg211-client: oleg211-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 02:09:41 (1780812581) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115238826936211 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 02:09:42.000000000 -0400 Modify: 2026-06-07 02:09:42.000000000 -0400 Change: 2026-06-07 02:09:42.000000000 -0400 Birth: 2026-06-07 02:09:42.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 02:09:54 (1780812594) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 02:09:56 (1780812596) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 02:09:58 (1780812598) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 02:09:59 (1780812599) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 02:10:01 (1780812601) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 02:10:03 (1780812603) total: 2500 open/close in 8.78 seconds: 284.88 ops/second snapshot_time 1780812635.936916152 secs.nsecs start_time 1780812604.976018533 secs.nsecs elapsed_time 30.960897619 secs.nsecs open 2505 samples [usecs] 156 12827 1443103 1114996989 close 2504 samples [usecs] 68 3627 378306 77493420 mknod 2501 samples [usecs] 227 12456 1284723 912780689 unlink 2500 samples [usecs] 376 67572 2751128 15890751422 mkdir 2 samples [usecs] 1210 1992 3202 5432164 rmdir 2 samples [usecs] 2891 4070 6961 24922781 rename 2500 samples [usecs] 314 8254 3612984 8073013536 getattr 3761 samples [usecs] 7 9544 125707 207098477 parallel_rename_file 2500 samples [usecs] 314 8254 3612984 8073013536 crossdir_rename 2500 samples [usecs] 314 8254 3612984 8073013536 PASS 81d (34s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 02:10:37 (1780812637) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7564 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7564 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 02:10:40 (1780812640) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 02:10:41 (1780812641) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 85: Lustre API root cache race =========== 02:10:49 (1780812649) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1780812650 Finishing test test1 at 1780812651 PASS 85 (4s) == sanityn test 90: open/create and unlink striped directory ========================================================== 02:10:53 (1780812653) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 02:10:54 (1780812654) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) == sanityn test 92: create remote directory under orphan directory ========================================================== 02:10:56 (1780812656) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 02:10:57 (1780812657) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6379 0x18eb 0x240000400 1 6344 0x18c8 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6380 0x18ec 0x240000400 1 6345 0x18c9 0x280000400 rc1=0 and rc2=0 PASS 93 (11s) == sanityn test 94: signal vs CP callback race =========== 02:11:08 (1780812668) osc.lustre-OST0000-osc-ffff8ab68e174800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8ab68eb16800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8ab68e174800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8ab68eb16800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0672428 s, 122 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09933 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6178: 292517 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0246964 s, 166 kB/s osc.lustre-OST0000-osc-ffff8ab68e174800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8ab68eb16800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ab68e174800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ab68eb16800.idle_timeout=debug PASS 94 (12s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 02:11:20 (1780812680) llite.lustre-ffff8ab68e174800.fast_read=0 llite.lustre-ffff8ab68eb16800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0233655 s, 89.8 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8ab68e174800.fast_read=1 llite.lustre-ffff8ab68eb16800.fast_read=1 PASS 95a (15s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 02:11:35 (1780812695) llite.lustre-ffff8ab68e174800.fast_read=0 llite.lustre-ffff8ab68eb16800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.0043868 s, 2.8 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8ab68e174800.fast_read=1 llite.lustre-ffff8ab68eb16800.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 02:11:44 (1780812704) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 02:11:45 (1780812705) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00197803 s, 2.1 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 02:11:48 (1780812708) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0863737 s, 24.3 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 02:11:51 (1780812711) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0637818 s, 32.9 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (3s) == sanityn test 100e: DoM: read on open and file size ==== 02:11:54 (1780812714) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00375432 s, 3.2 MB/s PASS 100e (4s) == sanityn test 101a: Discard DoM data on unlink ========= 02:11:58 (1780812718) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00269122 s, 1.5 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 02:12:01 (1780812721) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00209047 s, 2.0 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 02:12:03 (1780812723) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00203947 s, 2.0 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 02:12:08 (1780812728) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1780812729 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1780812732 Finishing test test23 at 1780812732 PASS 103 (9s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 02:12:17 (1780812737) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 02:12:18 (1780812738) fail_loc=0x416 PASS 105 (19s) == sanityn test 106a: Verify the btime via statx() ======= 02:12:37 (1780812757) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 02:12:38 (1780812758) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00376945 s, 272 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238826938746 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 02:12:39.000000000 -0400 Modify: 2026-06-07 02:12:39.000000000 -0400 Change: 2026-06-07 02:12:39.000000000 -0400 Birth: 2026-06-07 02:12:39.000000000 -0400 /mnt/lustre/f106b.sanityn 144115238826938746 -rw-r--r-- 1 0 0 1780812759 1780812759 1780812759 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238826938746 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 02:12:39.000000000 -0400 Modify: 2026-06-07 02:12:39.000000000 -0400 Change: 2026-06-07 02:12:39.000000000 -0400 Birth: 2026-06-07 02:12:39.000000000 -0400 1780812759 1024 1 PASS 106b (3s) == sanityn test 106c: Verify statx attributes mask ======= 02:12:41 (1780812761) PASS 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 02:12:44 (1780812764) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.13398 s, 78.3 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6351 0x18cf 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 02:12:49 (1780812769) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.113659 s, 92.3 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6389 0x18f5 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7564 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 02:12:59 (1780812779) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00308662 s, 3.2 MB/s llite.lustre-ffff8ab68e174800.hybrid_io=0 llite.lustre-ffff8ab68eb16800.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.29122 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 02:13:06 (1780812786) 192.168.202.111@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg211-client.virtnet /mnt/lustre (opts:) 192.168.202.111@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg211-client.virtnet /mnt/lustre2 (opts:) Starting race between client mounts (duration 30) Iteration 0 Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 192.168.202.111@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.111@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg211-client.virtnet /mnt/lustre (opts:) Stopping client oleg211-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg211-server' oleg211-server: oleg211-server.virtnet: executing load_modules_local oleg211-server: Loading modules from /home/green/git/lustre-release/lustre oleg211-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg211-server: detected 4 online CPUs by sysfs oleg211-server: MODOPTS_LIBCFS= oleg211-server: Force libcfs to create 2 CPU partitions Iteration 1 Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 192.168.202.111@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.111@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg211-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg211-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg211-server' oleg211-server: oleg211-server.virtnet: executing load_modules_local oleg211-server: Loading modules from /home/green/git/lustre-release/lustre oleg211-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg211-server: detected 4 online CPUs by sysfs oleg211-server: MODOPTS_LIBCFS= oleg211-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 192.168.202.111@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.111@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg211-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg211-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg211-server' oleg211-server: oleg211-server.virtnet: executing load_modules_local oleg211-server: Loading modules from /home/green/git/lustre-release/lustre oleg211-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg211-server: detected 4 online CPUs by sysfs oleg211-server: MODOPTS_LIBCFS= oleg211-server: Force libcfs to create 2 CPU partitions Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre2 PASS 109 (42s) == sanityn test 110: do not grant another lock on resend ========================================================== 02:13:48 (1780812828) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238860488707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-07 02:13:49.000000000 -0400 Modify: 2026-06-07 02:13:49.000000000 -0400 Change: 2026-06-07 02:13:49.000000000 -0400 Birth: 2026-06-07 02:13:50.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg211-client.virtnet oleg211-client.virtnet PASS 110 (166s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 02:16:34 (1780812994) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (2s) == sanityn test 112: update max-inherit in default LMV === 02:16:36 (1780812996) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 02:16:37 (1780812997) lustre-OST0000-osc-ffff8ab687aae000 active. lustre-OST0001-osc-ffff8ab687aae000 active. lustre-OST0000-osc-ffff8ab687aae000 active. lustre-OST0001-osc-ffff8ab687aae000 active. MGC192.168.202.111@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 02:16:39 (1780812999) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (2s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 02:16:41 (1780813001) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 02:16:42 (1780813002) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 121: trunc append race =================== 02:16:43 (1780813003) 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.00341681 s, 6.0 MB/s fail_loc=0x80001436 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0285945 s, 716 kB/s 0000000 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0005000 0000 0000 0000 0000 0000 0000 0000 0000 * 000c800 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0011800 PASS 121 (4s) == sanityn test 200: service remains healthy while able to process request ========================================================== 02:16:47 (1780813007) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=9 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0967 s, 0.2 kB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0996 s, 0.2 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.093 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1023 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0994 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1034 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1081 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1509 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1419 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1537 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1591 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1696 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1612 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1778 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2267 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2478 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2512 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2287 s, 0.1 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ab686464800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8ab686464800.ost_server_uuid in IDLE state after 0 sec PASS 200 (67s) cleanup: ====================================================== == sanityn test complete, duration 5190 sec ============== 02:17:55 (1780813075) === sanityn: start cleanup 02:17:55 (1780813075) === Stopping clients: oleg211-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg211-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 02:19:00 (1780813140) ===