-----============= acceptance-small: sanityn ============----- Mon Dec 22 02:24:54 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg218-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a skipping tests SLOW=no: 33a === sanityn: start setup 02:25:03 (1766388303) === Starting client oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Mount client oleg218-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Started clients oleg218-client.virtnet: 192.168.202.118@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg218-client.virtnet: executing check_config_client /mnt/lustre oleg218-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg218-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff92ff0323d000.idle_timeout=debug osc.lustre-OST0000-osc-ffff92ff0715c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff0323d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff0715c000.idle_timeout=debug disable quota as required oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanityn: finish setup 02:25:20 (1766388320) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134251 s, 78.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8194] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 02:25:21 (1766388321) PASS 0 (4s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 02:25:25 (1766388325) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 02:25:28 (1766388328) -rw-r--r-- 1 root root 0 Dec 22 02:25 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 02:25:31 (1766388331) -rw-r--r-- 1 root root 0 Dec 22 02:25 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 02:25:34 (1766388334) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 02:25:37 (1766388337) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 02:25:40 (1766388340) total 0 -rwxrwxrwx 1 root root 0 Dec 22 02:25 f2a -rwxrwxrwx 1 root root 0 Dec 22 02:25 f2b total 0 -rwxrwxrwx 1 root root 0 Dec 22 02:25 f2a -rwxrwxrwx 1 root root 0 Dec 22 02:25 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00740907 s, 69.1 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 02:25:43 (1766388343) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (3s) == sanityn test 2g: check blocks update on sync write ==== 02:25:46 (1766388346) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0806892 s, 26.0 MB/s /mnt/lustre/f2g.sanityn has 4096 blocks /mnt/lustre2/f2g.sanityn has 4096 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 02:25:49 (1766388349) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 02:25:51 (1766388351) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 02:25:55 (1766388355) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 02:25:58 (1766388358) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 02:26:00 (1766388360) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 02:26:03 (1766388363) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 02:26:06 (1766388366) PASS 9a (4s) == sanityn test 9b: append to striped sparse file ======== 02:26:10 (1766388370) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702588 s, 14.9 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0499623 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (7s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 02:26:17 (1766388377) 1+0 records in 1+0 records out 1 byte copied, 0.00453879 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00734921 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0160595 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00741647 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00781539 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0102382 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00677997 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00856021 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0160259 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0143448 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0133433 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0107135 s, 0.1 kB/s PASS 10a (8s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 02:26:25 (1766388385) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00980746 s, 313 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0799384 s, 51.2 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00852236 s, 360 kB/s PASS 10b (7s) == sanityn test 11: execution of file opened for write should return error ============================================================== 02:26:32 (1766388392) striped dir -i1 -c2 -H crush2 /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (8s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 02:26:40 (1766388400) warning: '-runtime' deprecated, use '-t runtime' instead - link 418 (time 1766388412.78 total 10.00 last 41.79) - link 912 (time 1766388422.79 total 20.01 last 49.35) - link 1513 (time 1766388432.80 total 30.03 last 60.00) - link 2084 (time 1766388442.80 total 40.03 last 57.10) - link 2746 (time 1766388452.81 total 50.04 last 66.13) - link 3514 (time 1766388462.82 total 60.04 last 76.75) - link 4151 (time 1766388472.82 total 70.05 last 63.68) - link 4763 (time 1766388482.83 total 80.06 last 61.12) - link 5501 (time 1766388492.84 total 90.07 last 73.73) total: 6360 link in 100.00 seconds: 63.60 ops/second using seed 3630793208 running for 100 seconds - stat 8724 (time 1766388414 ; total 11 ; last 11) - stat 10000 (time 1766388415 ; total 12 ; last 1) - stat 20000 (time 1766388421 ; total 18 ; last 6) - stat 30000 (time 1766388427 ; total 24 ; last 6) - stat 40000 (time 1766388433 ; total 30 ; last 6) - stat 50000 (time 1766388439 ; total 36 ; last 6) - stat 60000 (time 1766388445 ; total 42 ; last 6) - stat 70000 (time 1766388451 ; total 48 ; last 6) - stat 79959 (time 1766388462 ; total 59 ; last 11) - stat 80000 (time 1766388462 ; total 59 ; last 0) - stat 87359 (time 1766388473 ; total 70 ; last 11) - stat 90000 (time 1766388476 ; total 73 ; last 3) - stat 97460 (time 1766388487 ; total 84 ; last 11) - stat 100000 (time 1766388490 ; total 87 ; last 3) - stat 107784 (time 1766388501 ; total 98 ; last 11) total: 109493 stats in 100 seconds: 1094.930054 stats/second - unlinked 0 (time 1766388453 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile6360) error: No such file or directory total: 6359 unlinks in 91 seconds: 69.879120 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22652) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22654) - No such process PASS 12 (149s) == sanityn test 13: test directory page revocation ======= 02:29:09 (1766388549) striped dir -i1 -c2 -H crush2 /mnt/lustre/d13 f13 f13 PASS 13 (8s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 02:29:17 (1766388557) striped dir -i0 -c2 -H crush /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 23865 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (6s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 02:29:23 (1766388563) striped dir -i0 -c2 -H crush /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 24456 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (6s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 02:29:29 (1766388569) striped dir -i0 -c2 -H crush /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 25047 Terminated $DIR1/$tdir/sleep 60 PASS 14b (5s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 02:29:34 (1766388574) striped dir -i0 -c2 -H crush2 /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 25642 Terminated $DIR1/$tdir/sleep 60 PASS 14c (5s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 02:29:39 (1766388579) striped dir -i0 -c2 -H all_char /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 26237 Terminated $DIR1/$tdir/sleep 60 PASS 14d (6s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 02:29:45 (1766388585) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky Waiting for MDT destroys to complete 0 STRIPECOUNT=2 ORIGFREE=7207404 MAXFREE=2097152000 0 lustre-OST0000 avl=344 grnt=323 diff=21 limit=46331 FULL lustre-OST0001 avl=544 grnt=1043 diff=-499 limit=46331 FULL ERROR: dd blocks 7198983 != file size 7199004 LOG LOG2 file /tmp/oos2.log:dd: error writing '/mnt/lustre/oosfile': No space left on device /tmp/oos2.log:3596422+0 records in /tmp/oos2.log:3596421+0 records out /tmp/oos2.log:3682735104 bytes (3.7 GB, 3.4 GiB) copied, 481.829 s, 7.6 MB/s /tmp/oos2.log2:dd: error writing '/mnt/lustre2/oosfile2': No space left on device /tmp/oos2.log2:3602563+0 records in /tmp/oos2.log2:3602562+0 records out /tmp/oos2.log2:3689023488 bytes (3.7 GB, 3.4 GiB) copied, 481.695 s, 7.7 MB/s Waiting for MDT destroys to complete Waiting for MDT destroys to complete PASS 15 (523s) == sanityn test 16a: 2500 iterations of dual-mount fsx === 02:38:28 (1766389108) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0720158 s, 116 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0638192 s, 131 MB/s Chance of close/open is 1 in 50 Seed set to 9109 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn skipping zero length punch hole skipping zero size read fallocating to largest ever: 0x5390ae truncating to largest ever: 0x8f7329 fallocating to largest ever: 0x8fd796 fallocating to largest ever: 0x9c6941 fallocating to largest ever: 0x9ece58 fallocating to largest ever: 0xa00000 000300[0] 1766389113.746898 read 0x831daf thru 0x83bb7a (0x9dcc bytes) 000400[1] 1766389114.910728 trunc from 0x44bedd to 0x3033be (0x148b1e bytes) 000500[0] 1766389116.053451 mapread 0x562503 thru 0x56adf7 (0x88f5 bytes) 000600[1] 1766389117.274956 punch from 0x10a753 to 0x110421 (0x5ccf bytes) 000700[0] 1766389118.394368 trunc from 0x65c995 to 0x9871a6 (0x32a812 bytes) 000800[0] 1766389119.579540 trunc from 0x273c23 to 0x27acbf (0x709d bytes) 000900[0] 1766389120.732583 trunc from 0x199db4 to 0x19ea84 (0x4cd1 bytes) 001100[1] 1766389123.323752 write 0x7e3e00 thru 0x7ee11a (0xa31b bytes) 001400[0] 1766389127.255535 mapwrite 0x90bc95 thru 0x910658 (0x49c4 bytes) 001500[0] 1766389128.364694 write 0x9c8206 thru 0x9c9156 (0xf51 bytes) 001700[1] 1766389130.753266 trunc from 0x983085 to 0x927f68 (0x5b11c bytes) 001800[1] 1766389132.096069 fallocate 0x4dfd5f thru 0x4eb836 (0xbad8 bytes) 001900[1] 1766389133.844955 fallocate 0x8c3de8 thru 0x8c5164 (0x137d bytes) 002100[0] 1766389136.616533 fallocate 0x844308 thru 0x84b4aa (0x71a3 bytes) 002200[0] 1766389138.517384 write 0x14e246 thru 0x153139 (0x4ef4 bytes) 002300[1] 1766389140.271598 punch from 0x43657a to 0x4365ba (0x41 bytes) 002400[0] 1766389141.909944 fallocate 0x5b6179 thru 0x5be02c (0x7eb4 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9143 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fallocating to largest ever: 0x8c036c truncating to largest ever: 0x99d23b fallocating to largest ever: 0x9c771b 000100[1] 1766389144.563057 punch from 0x794fad to 0x79d7b4 (0x8808 bytes) 000200[0] 1766389145.532150 mapwrite 0x9c1000 thru 0x9c2882 (0x1883 bytes) truncating to largest ever: 0x9f155f 000400[1] 1766389147.322028 mapwrite 0x0b1000 thru 0x0bf41e (0xe41f bytes) truncating to largest ever: 0x9ff858 000600[0] 1766389149.276691 punch from 0x073fa9 to 0x07cb1d (0x8b75 bytes) 000700[0] 1766389150.413142 punch from 0x2dc67b to 0x2e0e10 (0x4796 bytes) 000900[1] 1766389152.505574 punch from 0x4732a0 to 0x47ada8 (0x7b09 bytes) 001000[0] 1766389153.592242 mapread 0x212000 thru 0x214955 (0x2956 bytes) 001100[1] 1766389154.667705 mapread 0x6e7000 thru 0x6f3d4b (0xcd4c bytes) 001200[0] 1766389155.751127 read 0x2d1000 thru 0x2dd672 (0xc673 bytes) 001300[0] 1766389156.782395 trunc from 0x0c7db7 to 0x0ce7e1 (0x6a2b bytes) 001400[0] 1766389157.826007 mapread 0x160000 thru 0x16ae91 (0xae92 bytes) 001500[1] 1766389158.964906 punch from 0x22ce59 to 0x22f1af (0x2357 bytes) 001600[0] 1766389160.079571 punch from 0x0e17a6 to 0x0ea5ea (0x8e45 bytes) 002000[1] 1766389164.435553 trunc from 0x12f0ba to 0x13d7bd (0xe704 bytes) 002100[1] 1766389165.815646 trunc from 0x8ec3d1 to 0x36f864 (0x57cb6c bytes) 002200[0] 1766389166.894619 punch from 0x2d4598 to 0x2d6b84 (0x25ed bytes) 002300[0] 1766389168.056492 trunc from 0x18e674 to 0x197d7a (0x9707 bytes) fallocating to largest ever: 0xa00000 002400[0] 1766389169.334656 trunc from 0x6087dc to 0x801d55 (0x1f957a bytes) 002500[1] 1766389170.378915 read 0x08d000 thru 0x09591b (0x891c bytes) All operations completed A-OK! PASS 16a (64s) == sanityn test 16b: 2500 iterations of dual-mount fsx at small size ========================================================== 02:39:32 (1766389172) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0769184 s, 109 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0633362 s, 132 MB/s Chance of close/open is 1 in 50 Seed set to 9173 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fallocating to largest ever: 0x4fbb8c truncating to largest ever: 0x88c65b truncating to largest ever: 0x94253b 000100[1] 1766389174.884260 read 0x0183a2 thru 0x018971 (0x5d0 bytes) fallocating to largest ever: 0x9acce8 fallocating to largest ever: 0x9c3380 000400[0] 1766389178.123456 punch from 0x0cbf76 to 0x0cd9e4 (0x1a6f bytes) truncating to largest ever: 0x9dadb3 fallocating to largest ever: 0x9e5a04 000500[0] 1766389179.186546 write 0x83d9d2 thru 0x83ed73 (0x13a2 bytes) fallocating to largest ever: 0x9ef268 000600[1] 1766389180.292338 trunc from 0x0fd837 to 0x0fe685 (0xe4f bytes) 000700[0] 1766389181.682831 read 0x94f1d8 thru 0x94f958 (0x781 bytes) fallocating to largest ever: 0x9f051b fallocating to largest ever: 0x9f77e5 000900[0] 1766389183.974756 fallocate 0x0a9d4a thru 0x0aaab1 (0xd68 bytes) 001100[0] 1766389186.232667 mapread 0x1de550 thru 0x1de769 (0x21a bytes) 001400[0] 1766389189.938129 write 0x32ea41 thru 0x32f4bd (0xa7d bytes) 001500[1] 1766389191.029735 trunc from 0x63bffd to 0x63c5ce (0x5d2 bytes) 001600[0] 1766389191.938630 punch from 0x194771 to 0x1963f0 (0x1c80 bytes) 001700[1] 1766389193.214417 mapwrite 0x9a0516 thru 0x9a1950 (0x143b bytes) 001800[0] 1766389194.303326 punch from 0x2358c9 to 0x23767b (0x1db3 bytes) 001900[1] 1766389195.266842 trunc from 0x54bd79 to 0x479630 (0xd2748 bytes) 002000[0] 1766389196.065951 mapread 0x485f36 thru 0x48603a (0x105 bytes) 002100[1] 1766389196.831931 read 0x3361f5 thru 0x3371dd (0xfe9 bytes) 002300[0] 1766389198.246392 read 0x4c785f thru 0x4c7ffb (0x79d bytes) 002400[0] 1766389199.051838 trunc from 0x46251c to 0x464126 (0x1c0b bytes) All operations completed A-OK! PASS 16b (29s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 02:40:01 (1766389201) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0613244 s, 137 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0519958 s, 161 MB/s Waiting for MDT destroys to complete Chance of close/open is 1 in 50 Seed set to 9212 fd 0: /mnt/lustre/f16c.sanityn fd 1: /mnt/lustre2/f16c.sanityn fallocating to largest ever: 0x59aed3 fallocating to largest ever: 0x6ed527 fallocating to largest ever: 0x819ff8 truncating to largest ever: 0x928603 truncating to largest ever: 0x98a91c 000100[1] 1766389214.181555 trunc from 0x84f3d8 to 0x966a43 (0x11766c bytes) truncating to largest ever: 0x9b80b1 000200[1] 1766389215.477957 read 0x46922a thru 0x4752a5 (0xc07c bytes) fallocating to largest ever: 0x9e565c truncating to largest ever: 0x9fe79d 000400[1] 1766389217.622143 punch from 0x6ec83c to 0x6f8bd2 (0xc397 bytes) 000600[0] 1766389220.640389 mapread 0x48c770 thru 0x49b0e7 (0xe978 bytes) 000700[1] 1766389221.585101 write 0x0f8412 thru 0x0fbfda (0x3bc9 bytes) 000800[1] 1766389222.704361 fallocate 0x667352 thru 0x6699c9 (0x2678 bytes) 000900[0] 1766389223.799790 trunc from 0x0bba0c to 0x0cb613 (0xfc08 bytes) 001000[0] 1766389224.905252 trunc from 0x26cfb4 to 0x27abfa (0xdc47 bytes) 001100[1] 1766389226.036176 punch from 0x1021fe to 0x10e38b (0xc18e bytes) 001200[1] 1766389227.063630 punch from 0x2d1769 to 0x2d551d (0x3db5 bytes) 001300[1] 1766389228.113844 write 0x45c096 thru 0x45f77f (0x36ea bytes) 001600[0] 1766389231.315910 fallocate 0x26cf5c thru 0x274ddd (0x7e82 bytes) 001700[1] 1766389232.314814 read 0x469a6b thru 0x472f3e (0x94d4 bytes) 001800[1] 1766389233.392730 fallocate 0x19ac91 thru 0x1a5641 (0xa9b1 bytes) 002000[0] 1766389235.604152 write 0x9bcdff thru 0x9c9904 (0xcb06 bytes) 002100[0] 1766389236.560749 punch from 0x439d24 to 0x43ce63 (0x3140 bytes) 002200[1] 1766389237.581671 trunc from 0x1c3bbd to 0x1c5f8b (0x23cf bytes) 002300[1] 1766389238.821868 mapwrite 0x33aea2 thru 0x3489ed (0xdb4c bytes) 002400[1] 1766389240.129755 trunc from 0x74cc2e to 0x4e0bf8 (0x26c035 bytes) 002500[0] 1766389241.027834 write 0x116f18 thru 0x121905 (0xa9ee bytes) All operations completed A-OK! PASS 16c (42s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 02:40:43 (1766389243) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0551158 s, 152 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0549339 s, 153 MB/s ldlm.namespaces.MGC192.168.202.118@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff92ff0323d000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff92ff0715c000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff92ff0323d000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff92ff0715c000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff92ff0715c000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff92ff0715c000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.83251 s, 57.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.784179 s, 134 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.16102 s, 90.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.77122 s, 59.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.974154 s, 108 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.3829 s, 75.8 MB/s PASS 16d (12s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 02:40:55 (1766389255) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.101773 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122745 s, 85.4 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 02:40:57 (1766389257) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 02:41:20 (1766389280) TIMEOUT PASS 16g (26s) == sanityn test 16h: mmap read after truncate file ======= 02:41:46 (1766389306) striped dir -i0 -c2 -H crush /mnt/lustre/d16h.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (6s) == sanityn test 16i: read after truncate file ============ 02:41:53 (1766389313) striped dir -i0 -c2 -H crush /mnt/lustre/d16i.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (7s) == sanityn test 16j: race dio with buffered i/o ========== 02:42:00 (1766389320) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.19197 s, 22.0 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.93181 s, 13.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.51064 s, 10.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.27658 s, 20.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.72376 s, 15.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.45053 s, 18.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.22787 s, 11.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.47963 s, 17.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.79369 s, 14.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.90459 s, 13.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.95324 s, 13.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.12956 s, 23.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.21072 s, 21.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.56364 s, 16.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.60059 s, 16.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.918146 s, 28.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.05203 s, 24.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.36827 s, 19.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.44105 s, 18.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (29s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 02:42:29 (1766389349) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fallocating to largest ever: 0x3428364 skipping zero size read fallocating to largest ever: 0x3428364 fallocating to largest ever: 0x3428364 skipping zero size read skipping zero size read fallocating to largest ever: 0x3428364 skipping zero size read fallocating to largest ever: 0x3428364 skipping zero size read truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Mon Dec 22 02:42:50 EST 2025 + fsx 2 done: rc=0 Mon Dec 22 02:42:50 EST 2025 + fsx 3 done: rc=0 Mon Dec 22 02:42:50 EST 2025 + fsx 4 done: rc=0 Mon Dec 22 02:42:50 EST 2025 + fsx 5 done: rc=0 Mon Dec 22 02:42:50 EST 2025 PASS 16k (26s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 02:42:55 (1766389375) fail_loc=0x8000030a PASS 17 (7s) == sanityn test 18: mmap sanity check =========================================================================================== 02:43:02 (1766389382) mmap test1: basic mmap operation (PASS, 0.06612s) mmap test2: MAP_PRIVATE not write back (PASS, 0.035032s) mmap test3: concurrent mmap ops on two nodes (PASS, 18.705s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.3817s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.11628s) mmap test6: check mmap write/read content on two nodes (PASS, 0.17402s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.72006s) mmap test11: multi-thread shared mmap (PASS, 0.70944s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0558s) PASS 18 (30s) == sanityn test 19: test concurrent uncached read races ========================================================================= 02:43:33 (1766389413) 32+0 records in 32+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.210982 s, 79.5 MB/s loop 5 loop 10 loop 15 loop 20 PASS 19 (16s) == sanityn test 20: test extra readahead page left in cache ============================================================== 02:43:49 (1766389429) striped dir -i0 -c2 -H crush /mnt/lustre/d20.sanityn PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 02:43:52 (1766389432) striped dir -i1 -c2 -H crush /mnt/lustre/d21.sanityn rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 02:43:56 (1766389436) now is 1766389437 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 new atime is 1766389498 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 02:45:01 (1766389501) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1792 1269512 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 8664 3598356 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 30288 3576732 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 38952 7175088 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 1792 1269512 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 8664 3598356 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 30288 3576732 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 38952 7175088 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 354 999.7K 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1000.0K 279 999.7K 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 256.0K 403 255.6K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 256.0K 402 255.6K 1% /mnt/lustre[OST:1] filesystem_summary: 511.8K 633 511.2K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 354 999.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1000.0K 279 999.7K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 256.0K 403 255.6K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 256.0K 402 255.6K 1% /mnt/lustre2[OST:1] filesystem_summary: 511.8K 633 511.2K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 1.3G 2.0M 1.2G 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1.3G 1.8M 1.2G 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3.7G 8.5M 3.4G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.7G 29.6M 3.4G 1% /mnt/lustre[OST:1] filesystem_summary: 7.3G 38.0M 6.8G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 354 1023646 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1024000 279 1023721 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 262144 403 261741 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 262144 402 261742 1% /mnt/lustre2[OST:1] filesystem_summary: 524116 633 523483 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1792 1269512 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 8664 3598356 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 30288 3576732 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 38952 7175088 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 354 999.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1000.0K 279 999.7K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 256.0K 403 255.6K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 256.0K 402 255.6K 1% /mnt/lustre2[OST:1] filesystem_summary: 511.8K 633 511.2K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 354 1023646 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1024000 279 1023721 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 262144 403 261741 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 262144 402 261742 1% /mnt/lustre[OST:1] filesystem_summary: 524116 633 523483 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 354 1023646 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1024000 279 1023721 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 262144 403 261741 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 262144 402 261742 1% /mnt/lustre2[OST:1] filesystem_summary: 524116 633 523483 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1792 1269512 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 8664 3598356 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 30288 3576732 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 38952 7175088 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 1792 1269512 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 8664 3598356 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 30288 3576732 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 38952 7175088 1% /mnt/lustre2 PASS 24a (4s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 02:45:05 (1766389505) PASS 24b (4s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 02:45:09 (1766389509) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (4s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 02:45:13 (1766389513) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] PASS 25b (5s) == sanityn test 26a: allow mtime to get older ============ 02:45:18 (1766389518) utime: good mknod times 1766389518 <= 1766389519 <= 1766389519 for /mnt/lustre/f26a utime: good mknod times 1766389518 <= 1766389519 <= 1766389519 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 02:45:23 (1766389523) PASS 26b (6s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 02:45:29 (1766389529) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289279613 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-12-22 02:45:30.000000000 -0500 Birth: 2025-12-22 02:45:30.000000000 -0500 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289279613 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-12-22 02:45:30.000000000 -0500 Birth: 2025-12-22 02:45:30.000000000 -0500 PASS 26c (6s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 02:45:36 (1766389536) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d30.sanityn 0 PASS 30 (6s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 02:45:42 (1766389542) striped dir -i1 -c2 -H crush2 /mnt/lustre/d31a.sanityn fail_loc=0x314 PASS 31a (6s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 02:45:48 (1766389548) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg218-server mds-ost sync done. Waiting for MDT destroys to complete striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d31b.sanityn fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.118@tcp:/lustre 7666232 38956 7175056 1% /mnt/lustre2 PASS 31b (12s) == sanityn test 31r: open-rename(replace) race =========== 02:46:00 (1766389560) -rw-r--r-- 1 root root 0 Dec 22 02:46 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (7s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 02:46:07 (1766389567) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 47586 Terminated tail -f $tfile PASS 31s (5s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 02:46:12 (1766389572) PASS 31t (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 02:46:18 (1766389578) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 02:46:20 (1766389580) Stopping /mnt/lustre-mds1 (opts:) on oleg218-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0000 create file on 2nd stripe under striped directory /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 2 /mnt/lustre/d33c.sanityn/striped touch /mnt/lustre2/d33c.sanityn/striped/subfile SoLC count 2 Below operations shouldn't trigger Solc: create file on 2nd stripe after setattr chmod 777 /mnt/lustre/d33c.sanityn/striped; touch /mnt/lustre2/d33c.sanityn/striped/subfile create remote dir and local dir /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33c.sanityn/remote create local dir after remote dir creation transaction commit /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33c.sanityn/remote.3; do_facet mds2 /home/green/git/lustre-release/lustre/utils/lctl set_param -n osd*.*MDT0001.force_sync 1; mkdir /mnt/lustre/d33c.sanityn/local.3 Solc count 0 PASS 33c (30s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 02:46:51 (1766389611) remote directory create /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre2/d33d.sanityn/subdir CoS count 2 cross-MDT rename mkdir /mnt/lustre/d33d.sanityn; /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn/d1; /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33d.sanityn/d2; touch /mnt/lustre/d33d.sanityn/d1/src mv /mnt/lustre2/d33d.sanityn/d1/src /mnt/lustre2/d33d.sanityn/d2/tgt CoS count 2 migrate /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn /home/green/git/lustre-release/lustre/utils/lfs migrate -m 1 /mnt/lustre2/d33d.sanityn CoS count 2 PASS 33d (26s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 02:47:17 (1766389637) plain dir creation mkdir /mnt/lustre2/d33e.sanityn/plain open file and write echo abc > /mnt/lustre2/d33e.sanityn/f33e.sanityn append write echo dfg >> /mnt/lustre2/d33e.sanityn/f33e.sanityn setattr touch /mnt/lustre2/d33e.sanityn/f33e.sanityn file unlink rm /mnt/lustre2/d33e.sanityn/f33e.sanityn plain dir unlink rmdir /mnt/lustre2/d33e.sanityn/plain striped directory creation /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 2 /mnt/lustre2/d33e.sanityn/striped set default LMV to create striped subdir /home/green/git/lustre-release/lustre/utils/lfs setdirstripe -D -c 2 /mnt/lustre/d33e.sanityn striped subdir creation createmany -d /mnt/lustre/d33e.sanityn/subdir 100 total: 100 mkdir in 0.96 seconds: 104.41 ops/second sub file creation and write createmany -o /mnt/lustre/d33e.sanityn/subfile 100; echo abc > /mnt/lustre/d33e.sanityn/subfile1 total: 100 open/close in 1.03 seconds: 96.96 ops/second sub file append write echo dfg >> /mnt/lustre2/d33e.sanityn/subfile2 subdir setatttr touch /mnt/lustre2/d33e.sanityn/subdir1 subdir unlink unlinkmany -d /mnt/lustre/d33e.sanityn/subdir 100 - unlinked 0 (time 1766389641 ; total 0 ; last 0) total: 100 unlinks in 4 seconds: 25.000000 unlinks/second sub file unlink unlinkmany /mnt/lustre2/d33e.sanityn/subfile 100 - unlinked 0 (time 1766389645 ; total 0 ; last 0) total: 100 unlinks in 2 seconds: 50.000000 unlinks/second sub file creation follows striped dir chmod chmod 777 /mnt/lustre/d33e.sanityn/striped; touch /mnt/lustre/d33e.sanityn/striped/subfile striped directory unlink rm -rf /mnt/lustre2/d33e.sanityn/striped directory unlink rm -rf /mnt/lustre2/d33e.sanityn CoS count 0 PASS 33e (13s) == sanityn test 34: no lock timeout under IO ============= 02:47:30 (1766389650) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92ff0323d000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff92ff0323d000.ost_server_uuid in IDLE state after 0 sec oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92ff0323d000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff92ff0323d000.ost_server_uuid in IDLE state after 0 sec oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92ff0323d000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff92ff0323d000.ost_server_uuid in IDLE state after 0 sec oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92ff0323d000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff92ff0323d000.ost_server_uuid in IDLE state after 0 sec oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92ff0323d000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff92ff0323d000.ost_server_uuid in IDLE state after 0 sec oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92ff0323d000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff92ff0323d000.ost_server_uuid in IDLE state after 0 sec PASS 34 (132s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 02:49:43 (1766389783) striped dir -i1 -c2 -H crush /mnt/lustre/d35.sanityn fail_loc=0x80000317 Race attempt 0 Wait for 57581 57605 for 60 sec... - open/close 811 (time 1766389800.10 total 10.02 last 80.96) - open/close 1684 (time 1766389810.10 total 20.02 last 87.30) - open/close 2639 (time 1766389820.12 total 30.03 last 95.35) - open/close 3613 (time 1766389830.12 total 40.04 last 97.37) total: 4000 open/close in 44.47 seconds: 89.95 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.118@tcp:/lustre 7.4G 40M 6.9G 1% /mnt/lustre 192.168.202.118@tcp:/lustre 7.4G 40M 6.9G 1% /mnt/lustre2 PASS 35 (72s) == sanityn test 36: handle ESTALE/open-unlink correctly == 02:50:55 (1766389855) striped dir -i0 -c2 -H crush /mnt/lustre/d36.sanityn Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg218-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.09271 s, 48.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(0) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.18469 s, 44.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(1) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.28089 s, 40.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(2) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.09384 s, 47.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(3) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.45956 s, 35.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(4) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.40603 s, 37.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(5) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.05198 s, 49.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(6) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.92009 s, 57.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(7) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.12305 s, 46.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(8) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.38566 s, 37.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(9) *** before(7173904) after_dd(7122704) after(7173904) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.12403 s, 46.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Waiting for MDT destroys to complete *** cycle(10) *** before(7173904) after_dd(7122704) after(7173904) PASS 36 (243s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 02:54:58 (1766390098) striped dir -i1 -c2 -H crush2 /mnt/lustre/d37.sanityn multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 - create 1789 (time 1766390111.56 total 10.00 last 178.83) - create 3676 (time 1766390121.56 total 20.00 last 188.68) - create 5603 (time 1766390131.57 total 30.01 last 192.53) - create 7476 (time 1766390141.57 total 40.02 last 187.23) - create 9141 (time 1766390151.57 total 50.02 last 166.44) total: 10000 create in 54.71 seconds: 182.77 ops/second PASS 37 (64s) == sanityn test 39a: file mtime does not change after rename ========================================================== 02:56:02 (1766390162) repeat after cancel_lru_locks PASS 39a (5s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 02:56:08 (1766390168) repeat after cancel_lru_locks PASS 39b (5s) == sanityn test 39c: check truncate mtime update ================================================================================ 02:56:13 (1766390173) repeat after cancel_lru_locks PASS 39c (6s) == sanityn test 39d: sync write should update mtime ====== 02:56:19 (1766390179) fail_loc=0x411 fail_loc=0 PASS 39d (5s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 02:56:25 (1766390185) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (14s) == sanityn test 40c: pdirops: link and others ======================================================================== 02:56:39 (1766390199) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (15s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 02:56:54 (1766390214) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (14s) == sanityn test 40e: pdirops: rename and others ======================================================================== 02:57:08 (1766390228) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (12s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 02:57:20 (1766390240) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (9s) == sanityn test 41b: pdirops: create vs create ======================================================================== 02:57:29 (1766390249) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 41b (11s) == sanityn test 41c: pdirops: create vs link ======================================================================== 02:57:40 (1766390260) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (11s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 02:57:51 (1766390271) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41d (9s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 02:58:00 (1766390280) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41e (10s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 02:58:10 (1766390290) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41f (11s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 02:58:21 (1766390301) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41g (11s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 02:58:32 (1766390312) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41h (10s) == sanityn test 41i: reint_open: create vs create ======== 02:58:42 (1766390322) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (867s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 03:13:09 (1766391189) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (7s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 03:13:16 (1766391196) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 open: File exists Conflict PASS 42b (7s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 03:13:23 (1766391203) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (7s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 03:13:30 (1766391210) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 42d (6s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 03:13:36 (1766391216) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (7s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 03:13:43 (1766391223) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 42f (7s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 03:13:50 (1766391230) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42g (7s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 03:13:57 (1766391237) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 42h (6s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 03:14:03 (1766391243) PASS 43a (44s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 03:14:47 (1766391287) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 Conflict PASS 43b (7s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 03:14:54 (1766391294) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 Conflict PASS 43c (7s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 03:15:01 (1766391301) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 03:15:07 (1766391307) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43e (7s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 03:15:14 (1766391314) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (7s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 03:15:21 (1766391321) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (7s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 03:15:28 (1766391328) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43h (6s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 03:15:34 (1766391334) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43i (7s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 03:15:41 (1766391341) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (63s) == sanityn test 43k: unlink vs create ==================== 03:16:44 (1766391404) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (842s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 03:30:46 (1766392246) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon Dec 22 03:30:51 EST 2025 PASS 44a (7s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 03:30:54 (1766392254) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 open: File exists Conflict PASS 44b (8s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 03:31:01 (1766392261) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (6s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 03:31:07 (1766392267) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 Conflict PASS 44d (7s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 03:31:15 (1766392275) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44e (8s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 03:31:22 (1766392282) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44f (7s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 03:31:29 (1766392289) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44g (8s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 03:31:37 (1766392297) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44h (7s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 03:31:44 (1766392304) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f44i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f44i.sanityn': File exists Conflict PASS 44i (8s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 03:31:52 (1766392312) PASS 45a (92s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 03:33:24 (1766392404) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45b (7s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 03:33:31 (1766392411) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45c (7s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 03:33:38 (1766392418) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (7s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 03:33:45 (1766392425) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 45e (8s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 03:33:53 (1766392433) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (8s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 03:34:01 (1766392441) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (8s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 03:34:09 (1766392449) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 45h (6s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 03:34:15 (1766392455) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 45i (8s) == sanityn test 45j: read vs rename ====================== 03:34:23 (1766392463) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (617s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 03:44:40 (1766393080) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (7s) == sanityn test 46b: pdirops: link vs create ======================================================================== 03:44:47 (1766393087) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 open: File exists Conflict PASS 46b (6s) == sanityn test 46c: pdirops: link vs link ======================================================================== 03:44:54 (1766393094) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 03:45:00 (1766393100) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46d (6s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 03:45:06 (1766393106) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46e (7s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 03:45:13 (1766393113) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 03:45:19 (1766393119) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46g (7s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 03:45:26 (1766393126) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46h (7s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 03:45:33 (1766393133) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f46i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f46i.sanityn': File exists Conflict PASS 46i (7s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 03:45:40 (1766393140) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f47a.sanityn': File exists Conflict PASS 47a (6s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 03:45:46 (1766393146) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 open: File exists Conflict PASS 47b (8s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 03:45:54 (1766393154) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f47c.sanityn' to '/mnt/lustre2/f47c.sanityn-2': File exists Conflict PASS 47c (6s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 03:46:00 (1766393160) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 Conflict PASS 47d (6s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 03:46:06 (1766393166) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f47e.sanityn' with non-directory Conflict PASS 47e (7s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 03:46:13 (1766393173) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 Conflict PASS 47f (6s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 03:46:19 (1766393179) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47g (8s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 03:46:27 (1766393187) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00684057 s, 1.5 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 03:46:34 (1766393194) 0+1 records in 0+1 records out 158 bytes copied, 0.00683691 s, 23.1 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3510: kill: (284494) - No such process PASS 51a (5s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 03:46:39 (1766393199) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00672714 s, 152 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245435 s, 42.7 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 03:46:54 (1766393214) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00423513 s, 242 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 03:47:01 (1766393221) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182217 s, 57.5 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 03:47:05 (1766393225) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1309 0x51d 0x2c0000401 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1309 0x51d 0x2c0000401 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 03:47:09 (1766393229) llite.lustre-ffff92ff0323d000.xattr_cache=1 llite.lustre-ffff92ff0715c000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762682 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:47:09.000000000 -0500 Modify: 2025-12-22 03:47:09.000000000 -0500 Change: 2025-12-22 03:47:09.000000000 -0500 Birth: 2025-12-22 03:47:09.000000000 -0500 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281271 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:47:15.000000000 -0500 Modify: 2025-12-22 03:47:15.000000000 -0500 Change: 2025-12-22 03:47:15.000000000 -0500 Birth: 2025-12-22 03:47:15.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762685 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:47:21.000000000 -0500 Modify: 2025-12-22 03:47:21.000000000 -0500 Change: 2025-12-22 03:47:21.000000000 -0500 Birth: 2025-12-22 03:47:20.000000000 -0500 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281272 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:47:20.000000000 -0500 Modify: 2025-12-22 03:47:21.000000000 -0500 Change: 2025-12-22 03:47:21.000000000 -0500 Birth: 2025-12-22 03:47:20.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762686 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:47:26.000000000 -0500 Modify: 2025-12-22 03:47:27.000000000 -0500 Change: 2025-12-22 03:47:27.000000000 -0500 Birth: 2025-12-22 03:47:26.000000000 -0500 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281274 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:47:26.000000000 -0500 Modify: 2025-12-22 03:47:27.000000000 -0500 Change: 2025-12-22 03:47:27.000000000 -0500 Birth: 2025-12-22 03:47:26.000000000 -0500 PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 03:47:34 (1766393254) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 03:47:42 (1766393262) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 03:47:50 (1766393270) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 03:48:03 (1766393283) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 03:48:14 (1766393294) fail_loc=0x155 We survived after AB/BA race test fail_loc=0x0 PASS 55e (14s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 03:48:27 (1766393307) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (14s) == sanityn test 55g: rename: race with trylock =========== 03:48:41 (1766393321) mdt.lustre-MDT0000.enable_rename_trylock=1 mdt.lustre-MDT0001.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre2/d55g.sanityn/b1/b2' to a subdirectory of itself, '/mnt/lustre2/d55g.sanityn/a1/a2/a3/a4/b2' We survived race with trylock test fail_loc=0x0 PASS 55g (15s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 03:48:56 (1766393336) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1766393337 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 12.2078 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (9s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 03:49:05 (1766393345) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1766393346 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 3.93788 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (36s) == sanityn test 60: Verify data_version behaviour ======== 03:49:41 (1766393381) striped dir -i0 -c2 -H crush2 /mnt/lustre/d60.sanityn PASS 60 (4s) == sanityn test 70a: cd directory && rm directory ======== 03:49:45 (1766393385) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 03:49:48 (1766393388) PASS 70b (4s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 03:49:51 (1766393391) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0064086 s, 10.2 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00561709 s, 11.7 MB/s File: /mnt/lustre2/f71a.sanityn Size: 262144 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289281314 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:49:51.000000000 -0500 Modify: 2025-12-22 03:49:52.000000000 -0500 Change: 2025-12-22 03:49:52.000000000 -0500 Birth: 2025-12-22 03:49:51.000000000 -0500 26627 extent 0 in offset 65536, length 65536 flags (0x80000000): NET extent 1 in offset 196608, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 2, sum of lengths 131072, expected sum 131072 26628 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00436034 s, 15.0 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00422873 s, 15.5 MB/s File: /mnt/lustre/f71a.sanityn Size: 262144 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289281314 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:49:51.000000000 -0500 Modify: 2025-12-22 03:49:52.000000000 -0500 Change: 2025-12-22 03:49:52.000000000 -0500 Birth: 2025-12-22 03:49:51.000000000 -0500 26629 extent 0 in offset 65536, length 65536 flags (0x80000000): NET extent 1 in offset 196608, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 2, sum of lengths 131072, expected sum 131072 26628 PASS 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 03:49:54 (1766393394) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00790759 s, 8.3 MB/s extent 0 in offset 0, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 1, sum of lengths 65536, expected sum 65536 PASS 71b (3s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 03:49:57 (1766393397) filefrag 1.47.3-wc2 (11-Nov-2025) check last flag for file with 291 extents 290: 296960.. 296963: 370688.. 370691: 4: 0001: last,net,eof check last flag for file with 292 extents 291: 297984.. 297987: 371712.. 371715: 4: 0000: last,net,eof check last flag for file with 293 extents 292: 299008.. 299011: 569344.. 569347: 4: 0001: last,net,eof PASS 71c (16s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 03:50:13 (1766393413) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00545242 s, 12.0 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0027601 s, 23.7 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00208973 s, 31.4 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0106286 s, 6.2 MB/s disk usage: 262144 /mnt/lustre/f71d.sanityn file size: 458752 /mnt/lustre/f71d.sanityn extent 0 in offset 0, length 65536 flags (0x80000000): NET extent 1 in offset 131072, length 65536 flags (0x80000000): NET extent 2 in offset 262144, length 65536 flags (0x80000000): NET extent 3 in offset 393216, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 4, sum of lengths 262144, expected sum 262144 PASS 71d (15s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 03:50:28 (1766393428) llite.lustre-ffff92ff0323d000.xattr_cache=1 llite.lustre-ffff92ff0715c000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 03:50:31 (1766393431) llite.lustre-ffff92ff0323d000.xattr_cache=1 llite.lustre-ffff92ff0715c000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 03:50:33 (1766393433) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00546275 s, 187 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00465718 s, 220 kB/s 299998: get lock2 300000: get lock1 299998: done 299998: get lock1 300000: done 300000: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 300000: cannot get lock2: Resource deadlock avoided 300000: exit rc=0 299998: done 299998: exit rc=0 PASS 74 (7s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 03:50:40 (1766393440) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0317993 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171248 s, 61.2 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4193: 300588 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (8s) == sanityn test 76: Verify MDT open_files listing ======== 03:50:48 (1766393448) striped dir -i0 -c2 -H all_char /mnt/lustre/d76.sanityn opening files: ................ closing files: ................ PASS 76 (59s) == sanityn test 77a: check FIFO NRS policy =============== 03:51:47 (1766393507) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.174953 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403166 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450259 s, 23.3 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889673 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926505 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105319 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102048 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108711 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132537 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14265 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134557 s, 7.8 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138974 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133802 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15801 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15451 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143069 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158054 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211907 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294526 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157944 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226397 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185117 s, 56.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189884 s, 55.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018835 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01694 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207249 s, 50.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241879 s, 43.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283396 s, 37.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247341 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273387 s, 38.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271597 s, 38.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029221 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320832 s, 32.7 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (4s) == sanityn test 77b: check CRR-N NRS policy ============== 03:51:51 (1766393511) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.189113 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032926 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229426 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465541 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580394 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125604 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131616 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13517 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120639 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126863 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137119 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150522 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143018 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149256 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117412 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126737 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372401 s, 28.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123902 s, 84.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187608 s, 55.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253199 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331201 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230768 s, 45.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195643 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264877 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03621 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268934 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280357 s, 37.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279878 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0548829 s, 19.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504822 s, 20.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037603 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329472 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174504 s, 6.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=6 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.18595 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740023 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885044 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943202 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080538 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836851 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114016 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115948 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111145 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911109 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152543 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146214 s, 7.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172331 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178369 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175186 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164093 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174733 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609783 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261295 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240828 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221433 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372095 s, 28.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328745 s, 31.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316022 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344966 s, 30.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270222 s, 38.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377252 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354309 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357936 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00488144 s, 215 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00513167 s, 204 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00500598 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0044614 s, 235 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 03:51:56 (1766393516) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.235561 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455472 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534023 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051465 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118811 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138826 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147481 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162488 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156667 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140492 s, 7.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168825 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165736 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15195 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155486 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165015 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156738 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1877 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107407 s, 97.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222081 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580658 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290516 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227509 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220447 s, 47.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345414 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284548 s, 36.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185272 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171735 s, 61.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234271 s, 44.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266521 s, 39.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221444 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221818 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402098 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450826 s, 23.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.186082 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414093 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769471 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107048 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104422 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104175 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920765 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990778 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0850033 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137637 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114785 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111677 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117234 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126587 s, 8.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130571 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144126 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127873 s, 8.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234282 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310471 s, 33.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181996 s, 57.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238553 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018914 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229697 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232766 s, 45.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016247 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202747 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245632 s, 42.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262006 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273056 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227349 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275899 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432189 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537136 s, 19.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.18658 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541372 s, 19.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0631705 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699938 s, 15.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986267 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102661 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855392 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919967 s, 11.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107556 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128017 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118449 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113926 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115281 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108697 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106874 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126583 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133998 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254014 s, 41.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173005 s, 60.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245779 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200381 s, 52.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192245 s, 54.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264625 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228585 s, 45.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275912 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242112 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290557 s, 36.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421038 s, 24.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302375 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324578 s, 32.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331662 s, 31.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274179 s, 38.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321683 s, 32.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 03:52:02 (1766393522) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.261603 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720828 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889583 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958732 s, 10.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807012 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105845 s, 9.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945891 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837353 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118073 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741372 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132335 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120674 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106269 s, 9.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130959 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126429 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124897 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130339 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263315 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300588 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288956 s, 36.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346371 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354667 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297944 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036561 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458702 s, 22.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315938 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040722 s, 25.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414577 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347054 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353085 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411184 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435785 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432849 s, 24.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.193161 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700945 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468045 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686204 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892734 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919637 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685059 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773331 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784672 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813238 s, 12.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152997 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14962 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151621 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155572 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160919 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169805 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150593 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230416 s, 45.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205069 s, 51.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236647 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209632 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021801 s, 48.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275128 s, 38.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244852 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239496 s, 43.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022506 s, 46.6 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=4 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207774 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180602 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232627 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201792 s, 52.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113962 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127671 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00273896 s, 383 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.178125 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573748 s, 18.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907341 s, 11.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647577 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11483 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112959 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104286 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124679 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939808 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117726 s, 8.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139466 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111309 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1225 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135344 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1326 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152083 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154679 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239707 s, 43.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618834 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00438097 s, 239 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312959 s, 33.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814452 s, 129 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124838 s, 84.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196755 s, 53.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192746 s, 54.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193498 s, 54.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265312 s, 39.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216726 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00395641 s, 265 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0040075 s, 262 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817043 s, 128 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00480073 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00501374 s, 209 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (6s) == sanityn test 77e: check TBF NID nrs policy ============ 03:52:08 (1766393528) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.18@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.254418 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379555 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979617 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0900539 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139734 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157405 s, 6.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186879 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205775 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199794 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196765 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199437 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211004 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197797 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220307 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188067 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214866 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195812 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271202 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250485 s, 41.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156692 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153664 s, 68.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232081 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312292 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456406 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381992 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227743 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272045 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026155 s, 40.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240345 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419969 s, 25.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037546 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298723 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374736 s, 28.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=2 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.216705 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546466 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710605 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687205 s, 15.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130837 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136311 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143736 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131063 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121025 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15707 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142105 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138912 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155786 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150147 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148872 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132931 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170914 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025101 s, 41.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227092 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240483 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325732 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350956 s, 29.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243738 s, 43.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293799 s, 35.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030726 s, 34.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325864 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435315 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331481 s, 31.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341748 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335824 s, 31.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038535 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454819 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432756 s, 24.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.190902 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862463 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680492 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856486 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861747 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768301 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922593 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109804 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115413 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130875 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150937 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143575 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148781 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147842 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161503 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143946 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16742 s, 6.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=6 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234255 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117227 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102892 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213359 s, 49.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270927 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163585 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234112 s, 44.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269868 s, 38.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025988 s, 40.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165331 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209281 s, 50.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783803 s, 134 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394102 s, 266 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269416 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029387 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171007 s, 61.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.205307 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377148 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510517 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066891 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050433 s, 20.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078099 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116307 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150033 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148135 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136243 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00476751 s, 220 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164048 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321732 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203012 s, 51.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144916 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224682 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674499 s, 155 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183796 s, 57.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168256 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355036 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123711 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173971 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044404 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038573 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158352 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425555 s, 24.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229178 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356895 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056984 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166554 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173957 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402042 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193859 s, 5.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 03:52:17 (1766393537) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.250446 s, 67.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045012 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0513864 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446418 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644752 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594345 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926166 s, 11.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146212 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122889 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121212 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123978 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141346 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150331 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16095 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139303 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135938 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12821 s, 8.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352375 s, 29.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298498 s, 35.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274466 s, 38.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363143 s, 28.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283811 s, 36.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412757 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153664 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334368 s, 31.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0418945 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354605 s, 29.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453092 s, 23.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571786 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424835 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365429 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365336 s, 28.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416556 s, 25.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.210035 s, 79.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409517 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487704 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833829 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800271 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887693 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816702 s, 12.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126011 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15857 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147275 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146915 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424612 s, 24.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17529 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355264 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289595 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153624 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323728 s, 32.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168377 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347107 s, 30.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224939 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328573 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231607 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181705 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376474 s, 27.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224832 s, 46.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168178 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168391 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394873 s, 26.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200202 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043934 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198309 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445449 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00319024 s, 329 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.235539 s, 71.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342992 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632053 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705146 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991358 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937661 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943693 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0915196 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134753 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119713 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148143 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145577 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160661 s, 6.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161003 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126945 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134937 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144705 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138546 s, 75.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136944 s, 76.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160481 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226901 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163351 s, 64.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017045 s, 61.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187871 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404469 s, 259 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303833 s, 34.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206228 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099988 s, 105 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201203 s, 52.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231092 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370541 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264458 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00375272 s, 279 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.194639 s, 86.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597669 s, 17.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120028 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944737 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118734 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114548 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129777 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123071 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121008 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13075 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137243 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149433 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139567 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150373 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114845 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15704 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143964 s, 7.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153169 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107896 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00511357 s, 205 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144808 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205974 s, 50.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259221 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00316232 s, 332 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462701 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00417772 s, 251 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166669 s, 62.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146887 s, 71.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146859 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147834 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111217 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562701 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087747 s, 119 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (10s) == sanityn test 77g: Change TBF type directly ============ 03:52:27 (1766393547) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 03:52:31 (1766393551) oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg218-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg218-client: oleg218-server: ssh exited with exit code 2 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg218-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg218-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg218-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 PASS 77h (5s) == sanityn test 77i: Change rank of TBF rule ============= 03:52:36 (1766393556) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 03:52:45 (1766393565) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.195842 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424243 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514242 s, 20.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485875 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348602 s, 30.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788168 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693419 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067951 s, 15.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694856 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816361 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0994396 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118614 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14266 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144803 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118082 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1454 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132859 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214768 s, 48.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198663 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160766 s, 65.2 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=7 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274739 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177197 s, 59.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03293 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018158 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198761 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195581 s, 53.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210075 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280954 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199906 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259376 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00357067 s, 294 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266007 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262857 s, 39.9 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93696 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.452 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (44s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 03:53:30 (1766393610) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.180413 s, 93.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428694 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404751 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524925 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741516 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720813 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975721 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101225 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0962212 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123582 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139335 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129702 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136757 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115853 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126782 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124372 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141009 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179681 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219883 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022961 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196093 s, 53.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234392 s, 44.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274561 s, 38.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276392 s, 37.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247589 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191803 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190114 s, 55.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025009 s, 41.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023064 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00414721 s, 253 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262755 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259871 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264264 s, 39.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6575 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9554 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.158356 s, 106 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0850506 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115667 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126348 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113439 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999631 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129165 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123484 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125083 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12043 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115853 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138821 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122363 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135514 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141272 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129712 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159345 s, 6.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183593 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194159 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194516 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204523 s, 51.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147318 s, 71.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206495 s, 50.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188436 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157271 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213526 s, 49.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220434 s, 47.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155174 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222293 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192222 s, 54.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212325 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244193 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246812 s, 42.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8304 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.971 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (113s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 03:55:23 (1766393723) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.177286 s, 94.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588466 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564321 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080142 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694105 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968497 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0729402 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999372 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111326 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128654 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118562 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139969 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11257 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136714 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128445 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147708 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157825 s, 6.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242528 s, 43.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290925 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243513 s, 43.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262545 s, 39.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334671 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337464 s, 31.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305016 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303803 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279093 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030273 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343427 s, 30.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407777 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268572 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337044 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366495 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357874 s, 29.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.659 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9661 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg218-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.15657 s, 107 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479236 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668973 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765031 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738859 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065523 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678974 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802855 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694648 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990437 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102629 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136058 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134098 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123572 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117081 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119051 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148227 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254277 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257058 s, 40.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023965 s, 43.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294867 s, 35.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306293 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227191 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028518 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197781 s, 53.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283297 s, 37.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205734 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00540403 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872022 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332185 s, 31.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136804 s, 76.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148708 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126061 s, 83.2 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6399 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.971 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (123s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 03:57:26 (1766393846) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.164408 s, 102 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067773 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947192 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12288 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118816 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122476 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113891 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121799 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118131 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105036 s, 10.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133674 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144244 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124532 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130927 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117336 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122725 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137958 s, 7.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016702 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190739 s, 55.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159723 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188544 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197342 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198114 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218571 s, 48.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241243 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902181 s, 116 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213421 s, 49.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111868 s, 93.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623874 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220022 s, 47.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201126 s, 52.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015216 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145508 s, 72.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=2 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91596 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73955 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.18@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.18@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.190213 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033369 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381281 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759283 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897752 s, 11.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123767 s, 8.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117936 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148104 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141115 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139414 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147917 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162192 s, 6.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152991 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157272 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151174 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16318 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00411972 s, 255 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157392 s, 66.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024432 s, 42.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173928 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490122 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157573 s, 66.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416374 s, 252 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00426411 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451646 s, 232 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00329553 s, 318 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250433 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556065 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252019 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419 s, 25.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352827 s, 29.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158982 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358398 s, 29.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91753 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7446 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.18@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.18947 s, 88.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039068 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559319 s, 18.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742463 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587793 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690878 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526977 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817649 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823361 s, 12.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110187 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0714143 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121092 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128312 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133809 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13976 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122843 s, 8.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131924 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141329 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185728 s, 56.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206759 s, 50.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225693 s, 46.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946757 s, 111 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123145 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136338 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204898 s, 51.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102006 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252591 s, 41.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644447 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198516 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208401 s, 50.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944363 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152645 s, 68.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187527 s, 55.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89807 s, 21.4 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.88459 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.163429 s, 103 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029548 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315601 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156412 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164329 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145191 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157964 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156067 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171527 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17241 s, 6.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169307 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17286 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156704 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157895 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166532 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162533 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166158 s, 6.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112145 s, 93.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264555 s, 39.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191868 s, 54.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207634 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195821 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223143 s, 47.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236846 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388317 s, 27.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260968 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254341 s, 41.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315248 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277694 s, 37.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032541 s, 32.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328378 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332856 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033317 s, 31.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75309 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76872 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.04706 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.90067 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.251115 s, 66.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609972 s, 17.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573532 s, 18.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739851 s, 14.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811968 s, 12.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089627 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656972 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811582 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907824 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.08106 s, 12.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727286 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123597 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156736 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166552 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101979 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114904 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142732 s, 7.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196006 s, 53.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197421 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00355395 s, 295 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0036625 s, 286 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00374216 s, 280 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00408215 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314237 s, 33.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266101 s, 39.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191149 s, 54.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047097 s, 22.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415544 s, 25.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380906 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451687 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184853 s, 56.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198933 s, 52.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183704 s, 57.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.528 s, 5.4 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4773 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.200025 s, 83.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341813 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401364 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787908 s, 13.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133823 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157377 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139689 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147579 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143208 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159868 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166766 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15913 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133183 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160564 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156192 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152861 s, 6.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=6 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287487 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296993 s, 35.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301268 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155225 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350687 s, 29.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0461821 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372109 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279288 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281873 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293989 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248634 s, 42.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575921 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424675 s, 24.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045544 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04545 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485031 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497899 s, 21.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94012 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76253 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.189339 s, 88.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331966 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685363 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795207 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0812813 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654162 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865116 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109297 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106886 s, 9.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125786 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109838 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144685 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136267 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150716 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14828 s, 7.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152558 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148505 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113252 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266783 s, 39.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237243 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201884 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148891 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163145 s, 64.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017265 s, 60.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171794 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250637 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195462 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201349 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220851 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225611 s, 46.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240043 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236215 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279935 s, 37.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.80612 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78836 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79423 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79591 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (272s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 04:01:58 (1766394118) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.192704 s, 87.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312199 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668147 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0599893 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985767 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101773 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864687 s, 12.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.058986 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126716 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104331 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120407 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117751 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127353 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114521 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124323 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11151 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139824 s, 7.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695144 s, 151 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116725 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013496 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037496 s, 28.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144639 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326842 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163731 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186656 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200498 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320687 s, 32.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141588 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218047 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227035 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226171 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241449 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255767 s, 41.0 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91681 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74898 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn Waiting for MDT destroys to complete PASS 77kb (30s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 04:02:28 (1766394148) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.18@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.18@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.183854 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428729 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.08179 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10685 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098064 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129402 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120233 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117213 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124879 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158328 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152774 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147724 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15213 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164578 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152887 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157806 s, 6.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329687 s, 31.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319514 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360028 s, 29.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278554 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164355 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00435772 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202532 s, 51.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364949 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307814 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00449249 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435361 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169285 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604455 s, 173 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506701 s, 20.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00440037 s, 238 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015306 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130363 s, 80.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=7 removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90465 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73401 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn Waiting for MDT destroys to complete PASS 77kc (26s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 04:02:54 (1766394174) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.18@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.152544 s, 110 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395157 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047915 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484557 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063045 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523025 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644837 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562725 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551936 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948256 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620367 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122426 s, 8.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111757 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111994 s, 9.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127307 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116666 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113454 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164591 s, 63.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150982 s, 69.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167677 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024342 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182923 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021874 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199522 s, 52.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939069 s, 112 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238164 s, 44.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223327 s, 47.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652886 s, 161 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649027 s, 162 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119578 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141644 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063113 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121923 s, 86.0 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89369 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97104 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn Waiting for MDT destroys to complete PASS 77kd (16s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 04:03:10 (1766394190) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.161588 s, 104 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338617 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473355 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760173 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809799 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856031 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684256 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848376 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793083 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078088 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086926 s, 12.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112855 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924571 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145534 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13203 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14508 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135163 s, 7.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117157 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125166 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228443 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667073 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250585 s, 41.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154604 s, 67.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840727 s, 125 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011767 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234817 s, 44.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217364 s, 48.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931816 s, 113 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182219 s, 57.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188091 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00268966 s, 390 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00298711 s, 351 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180205 s, 58.2 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75381 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74253 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89208 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73666 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn Waiting for MDT destroys to complete PASS 77ke (61s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 04:04:11 (1766394251) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.202778 s, 82.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300081 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328118 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133149 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111745 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114398 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111991 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111876 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137553 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123172 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137615 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151058 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131688 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146969 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143741 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127267 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132891 s, 7.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199026 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739048 s, 142 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962169 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250713 s, 41.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302598 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019159 s, 54.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253515 s, 41.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147067 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188045 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161099 s, 65.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888246 s, 118 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753158 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00518692 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00533881 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00494471 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345787 s, 30.3 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4646 s, 5.4 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9511 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn Waiting for MDT destroys to complete PASS 77kf (50s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 04:05:01 (1766394301) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.15572 s, 108 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386019 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537684 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0469571 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639934 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583879 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804496 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682875 s, 15.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0780856 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661303 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0962677 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095157 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963247 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125174 s, 8.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134962 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128182 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117908 s, 8.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00372459 s, 282 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160343 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998398 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131159 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141236 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177497 s, 59.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159573 s, 65.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172141 s, 60.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0031096 s, 337 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173185 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854105 s, 123 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00347179 s, 302 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119834 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159694 s, 65.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890716 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593343 s, 177 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89596 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73689 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.17888 s, 93.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561176 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429456 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143616 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229397 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250488 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24151 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232408 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236369 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231004 s, 4.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262008 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271197 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232091 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240009 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252598 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251776 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24842 s, 4.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189655 s, 55.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185464 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224075 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254915 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027148 s, 38.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320128 s, 32.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328592 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347468 s, 30.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109272 s, 96.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229088 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201614 s, 52.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188306 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00449198 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165691 s, 63.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010928 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197503 s, 53.1 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7503 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73744 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74012 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73307 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete PASS 77kg (89s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 04:06:30 (1766394390) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 04:06:33 (1766394393) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg218-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.72905 s, 281 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg218-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0054 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 04:07:24 (1766394444) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.146669 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304632 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587568 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0570384 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060723 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910806 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110178 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116361 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13018 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140518 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119782 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147558 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153372 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132101 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126005 s, 8.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154793 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167157 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016261 s, 64.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210366 s, 49.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299008 s, 35.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194809 s, 53.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289403 s, 36.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180305 s, 58.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194061 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139585 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319258 s, 32.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586891 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026204 s, 40.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255613 s, 41.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330644 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033004 s, 31.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386992 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308275 s, 34.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=1 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90546 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97201 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.129215 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483355 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382234 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877803 s, 11.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195386 s, 5.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203761 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197654 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214402 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190527 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189133 s, 5.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194537 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216746 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20746 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225966 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223497 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207316 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239763 s, 4.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff92ff0323d000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff92ff0323d000.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105097 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100741 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10237 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101223 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102851 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105625 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116587 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121354 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104066 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105174 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109288 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108096 s, 9.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109183 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109436 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113463 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129667 s, 8.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg218-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg218-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89402 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97057 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (47s) == sanityn test 77o: Changing rank should not panic ====== 04:08:11 (1766394491) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 04:08:15 (1766394495) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (38s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 04:08:53 (1766394533) mds.MDS.mdt.nrs_policies=tbf oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg218-client: oleg218-server: ssh exited with exit code 36 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg218-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (12s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 04:09:05 (1766394545) striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d77r.sanityn mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 338467 kill md_thread xargs PASS 77r (41s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 04:09:46 (1766394586) oleg218-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg218-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg218-client: oleg218-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 04:09:49 (1766394589) striped dir -i1 -c2 -H crush2 /mnt/lustre/d79.sanityn File: /mnt/lustre/d79.sanityn Size: 8192 Blocks: 16 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798763922 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:09:49.000000000 -0500 Modify: 2025-12-22 04:09:49.000000000 -0500 Change: 2025-12-22 04:09:49.000000000 -0500 Birth: 2025-12-22 04:09:49.000000000 -0500 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 04:10:01 (1766394601) total: 10 open/close in 0.03 seconds: 295.01 ops/second multiop /mnt/lustre2/d80a.sanityn/f80a.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Finish migration, then checking.. PASS 80a (4s) == sanityn test 80b: Accessing directory during migration ========================================================== 04:10:05 (1766394605) start migration thread 346947 accessing the migrating directory for 1 minute... concurrent access 1887 failures, 2640 successes /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 346947 Killed ( while true; do mdt_idx=$((RANDOM % MDSCOUNT)); $LFS migrate -m $mdt_idx $migdir1 &> /dev/null; done ) (wd: ~) PASS 80b (61s) == sanityn test 81a: rename and stat under striped directory ========================================================== 04:11:06 (1766394666) stat: cannot statx 'd0/0': No such file or directory File: d0/0 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289284197 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:11:07.000000000 -0500 Modify: 2025-12-22 04:11:07.000000000 -0500 Change: 2025-12-22 04:11:07.000000000 -0500 Birth: 2025-12-22 04:11:07.000000000 -0500 PASS 81a (3s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 04:11:09 (1766394669) total: 2500 open/close in 5.98 seconds: 417.75 ops/second /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 353462 Killed ( while true; do touch $DIR1/$tdir; done ) (wd: ~) PASS 81b (46s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 04:11:55 (1766394715) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 04:11:56 (1766394716) total: 2500 open/close in 4.62 seconds: 541.10 ops/second total: 2500 open/close in 5.63 seconds: 443.72 ops/second snapshot_time 1766394757.357833939 secs.nsecs start_time 1766394716.891656715 secs.nsecs elapsed_time 40.466177224 secs.nsecs open 2505 samples [usecs] 82 1708 601988 152434944 close 2504 samples [usecs] 44 1071 161664 11986054 mknod 2501 samples [usecs] 129 1152 506358 107861894 unlink 2500 samples [usecs] 223 15848 1218089 1381602721 mkdir 2 samples [usecs] 677 1994 2671 4434365 rmdir 2 samples [usecs] 1697 2362 4059 8458853 rename 2500 samples [usecs] 223 5531 3278531 6468359127 getattr 3764 samples [usecs] 1 1879 124780 8773380 statfs 8 samples [usecs] 9 60 170 5552 parallel_rename_file 2500 samples [usecs] 223 5531 3278531 6468359127 crossdir_rename 2500 samples [usecs] 223 5531 3278531 6468359127 snapshot_time 1766394757.357921994 secs.nsecs start_time 1766394716.891700565 secs.nsecs elapsed_time 40.466221429 secs.nsecs open 2504 samples [usecs] 76 1145 788790 260904706 close 2504 samples [usecs] 52 1715 204632 20551240 mknod 2500 samples [usecs] 157 1095 665666 187021658 unlink 2500 samples [usecs] 224 29207 1525340 4734210054 mkdir 2 samples [usecs] 870 5423 6293 30165829 rmdir 2 samples [usecs] 2273 7523 9796 61762058 rename 2500 samples [usecs] 220 7962 2841962 4593053970 getattr 3757 samples [usecs] 9 31642 163217 1385523893 statfs 8 samples [usecs] 9 33 154 3360 parallel_rename_file 2500 samples [usecs] 220 7962 2841962 4593053970 crossdir_rename 2500 samples [usecs] 220 7962 2841962 4593053970 PASS 81d (43s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 04:12:39 (1766394759) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.8194 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.8194 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 04:12:41 (1766394761) start pid 367876 to create/unlink striped directory start pid 367877 to stat striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5797: 367876 Terminated ( cd $DIR1; while true; do $LFS mkdir -i1 -c2 $tdir; rmdir $tdir; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5797: 367877 Terminated ( cd $DIR2; while true; do stat $tdir > /dev/null 2>&1; done ) PASS 83 (122s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 04:14:43 (1766394883) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 85: Lustre API root cache race =========== 04:14:51 (1766394891) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1766394892 Finishing test test1 at 1766394892 PASS 85 (3s) == sanityn test 90: open/create and unlink striped directory ========================================================== 04:14:54 (1766394894) start pid 378934 to open/create under striped directory start pid 378935 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5864: 378934 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; touch $tdir/f{0..3} > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5864: 378935 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 90 (182s) == sanityn test 91: chmod and unlink striped directory === 04:17:56 (1766395076) start pid 392051 to chmod striped directory start pid 392052 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5901: 392051 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; chmod go+w $tdir > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5901: 392052 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 91 (182s) == sanityn test 92: create remote directory under orphan directory ========================================================== 04:20:59 (1766395259) fail_loc=0x1408 mkdir: cannot create directory '/mnt/lustre2/d92.sanityn/dir': No such file or directory lfs setdirstripe: cannot resolve path '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory (2) lfs setdirstripe: '/mnt/lustre2/d92.sanityn/remote_dir' is not on a Lustre filesystem: No such file or directory (2) lfs setdirstripe: cannot create dir '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory fail_loc=0 PASS 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 04:21:01 (1766395261) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6483 0x1953 0x2c0000401 0 6500 0x1964 0x280000401 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6484 0x1954 0x2c0000401 0 6501 0x1965 0x280000401 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 04:21:10 (1766395270) osc.lustre-OST0000-osc-ffff92ff0323d000.idle_timeout=0 osc.lustre-OST0000-osc-ffff92ff0715c000.idle_timeout=0 osc.lustre-OST0001-osc-ffff92ff0323d000.idle_timeout=0 osc.lustre-OST0001-osc-ffff92ff0715c000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0125108 s, 655 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09979 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5988: 410921 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00518195 s, 790 kB/s osc.lustre-OST0000-osc-ffff92ff0323d000.idle_timeout=debug osc.lustre-OST0000-osc-ffff92ff0715c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff0323d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff0715c000.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 04:21:20 (1766395280) llite.lustre-ffff92ff0323d000.fast_read=0 llite.lustre-ffff92ff0715c000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.017235 s, 122 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff92ff0323d000.fast_read=1 llite.lustre-ffff92ff0715c000.fast_read=1 PASS 95a (15s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 04:21:35 (1766395295) llite.lustre-ffff92ff0323d000.fast_read=0 llite.lustre-ffff92ff0715c000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00165379 s, 7.4 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff92ff0323d000.fast_read=1 llite.lustre-ffff92ff0715c000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 04:21:43 (1766395303) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 04:21:44 (1766395304) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00185815 s, 2.2 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 04:21:47 (1766395307) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0222591 s, 94.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 04:21:49 (1766395309) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0187556 s, 112 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 04:21:51 (1766395311) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00206478 s, 5.8 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 04:21:53 (1766395313) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000766709 s, 5.3 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 04:21:56 (1766395316) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000575055 s, 7.1 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 04:21:58 (1766395318) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000521245 s, 7.9 MB/s PASS 101c (3s) == sanityn test 102: Test open by handle of unlinked file ========================================================== 04:22:01 (1766395321) file: /mnt/lustre/f102.sanityn fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 00 02 00 00 00 1c 33 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x200000402:0x331c:0x0] access via mount point '/mnt/lustre2' - OK access after unlink - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.remote fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 40 02 00 00 00 4b 3d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000402:0x3d4b:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.remote/subdir fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 40 02 00 00 00 4c 3d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000402:0x3d4c:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.stripe fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 00 02 00 00 00 1d 33 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x200000402:0x331d:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.stripe/subdir fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 00 02 00 00 00 1e 33 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x200000402:0x331e:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! PASS 102 (3s) == sanityn test 103: Test size correctness with lockahead ========================================================== 04:22:04 (1766395324) striped dir -i1 -c2 -H all_char /mnt/lustre/d103.sanityn fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1766395325 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1766395327 Finishing test test23 at 1766395328 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 04:22:11 (1766395331) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00261316 s, 392 kB/s oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395333 1766395333 1766395333 MDT a|m|ctime 1766395333 1766395333 1766395333 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00269957 s, 379 kB/s oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395333 1766395335 1766395335 MDT a|m|ctime 1766395333 1766395335 1766395335 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00708995 s, 144 kB/s oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395333 1766395339 1766395339 MDT a|m|ctime 1766395333 1766395339 1766395339 oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395333 1766395339 1766395339 MDT a|m|ctime 1766395333 1766395339 1766395339 oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395333 1766395339 1766395348 MDT a|m|ctime 1766395333 1766395339 1766395348 PASS 104 (19s) == sanityn test 105: Glimpse and lock cancel race ======== 04:22:31 (1766395351) striped dir -i1 -c2 -H crush2 /mnt/lustre/d105.sanityn fail_loc=0x416 PASS 105 (23s) == sanityn test 106a: Verify the btime via statx() ======= 04:22:53 (1766395373) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00281754 s, 363 kB/s oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 5147 Type: regular Mode: 0644 Flags: 0x80000 Generation: 1890547827 Version: 0x00000002:00017b46 User: 0 Group: 0 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69490dee:00000000 -- Mon Dec 22 04:22:54 2025 atime: 0x69490dee:00000000 -- Mon Dec 22 04:22:54 2025 mtime: 0x69490dee:00000000 -- Mon Dec 22 04:22:54 2025 crtime: 0x69490dee:1298be00 -- Mon Dec 22 04:22:54 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200000402:0x3322:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 22 33 00 00 00 00 00 00 02 04 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 c0 02 00 00 00 5a 19 00 00 00 00 00 00 00 00 00 00 01 00 00 00 user.job (4) = "dd.0" linkea: idx=0 parent=[0x200000007:0x1:0x0] name='f106a.sanityn' trusted.som (24) = 04 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EXTENTS: PASS 106a (3s) == sanityn test 106b: Glimpse RPCs test for statx ======== 04:22:56 (1766395376) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00347578 s, 295 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 8 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289292579 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:22:56.000000000 -0500 Modify: 2025-12-22 04:22:56.000000000 -0500 Change: 2025-12-22 04:22:56.000000000 -0500 Birth: 2025-12-22 04:22:56.000000000 -0500 /mnt/lustre/f106b.sanityn 144115205289292579 -rw-r--r-- 1 0 0 1766395376 1766395376 1766395376 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 8 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289292579 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:22:56.000000000 -0500 Modify: 2025-12-22 04:22:56.000000000 -0500 Change: 2025-12-22 04:22:56.000000000 -0500 Birth: 2025-12-22 04:22:56.000000000 -0500 1766395376 1024 8 PASS 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 04:22:58 (1766395378) PASS 106c (2s) == sanityn test 107a: Basic grouplock conflict =========== 04:23:00 (1766395380) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0913089 s, 115 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6508 0x196c 0x280000401 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 04:23:05 (1766395385) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.106649 s, 98.3 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6492 0x195c 0x2c0000401 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.8194 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 04:23:13 (1766395393) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00166182 s, 6.0 MB/s llite.lustre-ffff92ff0323d000.hybrid_io=0 llite.lustre-ffff92ff0715c000.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.18239 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 04:23:19 (1766395399) 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre 192.168.202.118@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.202.118@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 PASS 109 (507s) == sanityn test 110: do not grant another lock on resend ========================================================== 04:31:46 (1766395906) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934275 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:31:47.000000000 -0500 Modify: 2025-12-22 04:31:47.000000000 -0500 Change: 2025-12-22 04:31:47.000000000 -0500 Birth: 2025-12-22 04:31:46.000000000 -0500 fail_loc=0 fail_val=0 Connected clients: oleg218-client.virtnet oleg218-client.virtnet PASS 110 (65s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 04:32:51 (1766395971) fail_loc=0x8000018a /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6784: 483598 Terminated $MULTIOP $DIR2/$tdir/mdt0dir/foodir/file2 Ow4096_c PASS 111 (6s) == sanityn test 112: update max-inherit in default LMV === 04:32:57 (1766395977) lmv.lustre-clilmv-ffff92ff04cb9800.qos_threshold_rr=90 lmv.lustre-clilmv-ffff92ff37199000.qos_threshold_rr=90 PASS 112 (3s) == sanityn test 113: check servers of specified fs ======= 04:33:00 (1766395980) lustre-OST0000-osc-ffff92ff37199000 active. lustre-OST0001-osc-ffff92ff37199000 active. lustre-OST0000-osc-ffff92ff37199000 active. lustre-OST0001-osc-ffff92ff37199000 active. MGC192.168.202.118@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 04:33:02 (1766395982) /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1 lmv_stripe_count: 2 lmv_stripe_offset: 2 lmv_hash_type: none lmv_max_inherit: 2 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1 lmv_stripe_count: 2 lmv_stripe_offset: 2 lmv_hash_type: none lmv_max_inherit: 2 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 PASS 114 (7s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 04:33:09 (1766395989) total: 3000 create in 3.13 seconds: 959.25 ops/second oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: htree_dump_int_block: Ext2 directory block not found while calculating checksum for htree root oleg218-server: USE: f1013 f1446 fail_loc=0x80002401 fail_val=5 mkdir: cannot create directory '/mnt/lustre/d115.sanityn/f1446': File exists PASS 115 (13s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 04:33:22 (1766396002) llite.lustre-ffff92ff04cb9800.intent_mkdir=1 llite.lustre-ffff92ff37199000.intent_mkdir=1 MD layout /mnt/lustre/d116.sanityn: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x80e:0x0] 1 [0x240000404:0x787:0x0] mkdir /mnt/lustre/d116.sanityn/tdir0 setdirstripe -D -i1 /mnt/lustre2/d116.sanityn/tdir0 mkdir /mnt/lustre/d116.sanityn/tdir0/tdir11 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none mdtidx FID[seq:oid:ver] lmv_stripe_count: 0 lmv_stripe_offset: 1 lmv_hash_type: none mdtidx FID[seq:oid:ver] PASS 116 (5s) == sanityn test 117a: TCU: Init and enable Trash Can on MDTs ========================================================== 04:33:27 (1766396007) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 PASS 117a (6s) == sanityn test 117b: Move regular file and empty dir into trash can dir ========================================================== 04:33:34 (1766396014) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain llite.lustre-ffff92ff04cb9800.trash_file_read=1 llite.lustre-ffff92ff37199000.trash_file_read=1 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.148255 s, 28.3 MB/s /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0x200000bd1:0xbcc:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/trash/MDT0001: llite.lustre-ffff92ff04cb9800.trash_file_read=0 llite.lustre-ffff92ff37199000.trash_file_read=0 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117b (12s) == sanityn test 117c: Move deleted tree with multiple levels into trash ========================================================== 04:33:45 (1766396025) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: tdir1 tdir2 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir1: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir1/tdir11: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir1/tdir12: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir2: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117c (8s) == sanityn test 117d: Per-User Trash can Type testing ==== 04:33:53 (1766396033) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2/tdir21] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2/tdir22] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117d.sanityn/tdir2/tfile23] uid1: 0 uid2: 500 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: 0x200000bd1:0xbdb:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200000bd1:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/trash/MDT0000/0/0x200000bd1:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/trash/MDT0000/0/0x200000bd1:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: 0x200000bd1:0xbdf:0x0 /mnt/lustre/.lustre/trash/MDT0000/500/0x200000bd1:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/trash/MDT0000/500/0x200000bd1:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/trash/MDT0000/500/0x200000bd1:0xbdf:0x0/tdir22: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117d (5s) == sanityn test 117e: Undeleted dir in trash should keep its original xattrs ========================================================== 04:33:58 (1766396038) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/trash/MDT0000/500: trusted.name11="value11" trusted.name12="value12" trusted.name1="value1" mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117e (5s) == sanityn test 117f: Uncache the dentry under the trash dir ========================================================== 04:34:03 (1766396043) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/500: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: tdir1 tdir2 tfile3 tfile4 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir2: /mnt/lustre/.lustre/trash/MDT0000/500: tdir2 tfile4 -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: -u------------e------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tfile3 -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1: -u------------e------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tfile4 -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir2 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir2: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0 /mnt/lustre/.lustre/trash/MDT0000/0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/500 /mnt/lustre/.lustre/trash/MDT0000/500: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117f (5s) == sanityn test 117g: Access .Trash for a non-striped directory ========================================================== 04:34:08 (1766396048) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/500: ==> check '.Trash': tdir1 tfile3 <== tdir1 tfile3 tdir1 tfile3 tdir21 tfile22 tdir2 tdir21 tfile22 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117g (5s) == sanityn test 117h: Permission testing of various operations for files in Trash Can ========================================================== 04:34:13 (1766396053) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Trash list: ===> /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/500: <=== touch: setting times of '/mnt/lustre/.lustre/trash/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/trash/f117h.sanityn.crfail': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.crfail': Operation not permitted rmdir: failed to remove '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/f117h.sanityn.crfail': No such file or directory mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/d117h.sanityn.mvfail': Operation not permitted chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chmod: changing permissions of '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3': Bad file descriptor truncate: cannot truncate '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3' to length 4096: Bad file descriptor chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chmod: changing permissions of '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1': Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1: Bad file descriptor open: Bad file descriptor open: Bad file descriptor read: Operation not permitted mmap: Operation not permitted open: Is a directory open: Is a directory mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117h (5s) == sanityn test 117i: Forbid to create a file with name .Trash ========================================================== 04:34:18 (1766396058) mkdir: cannot create directory '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument touch: setting times of '/mnt/lustre/d117i.sanityn/.Trash': No such file or directory open(O_RDWR|O_CREAT): Invalid argument mcreate: cannot create `/mnt/lustre/d117i.sanityn/.Trash' with mode 0100644: Invalid argument ln: failed to create hard link '/mnt/lustre/d117i.sanityn/.Trash' => '/mnt/lustre/d117i.sanityn/f117i.sanityn': Invalid argument ln: failed to create symbolic link '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument mv: cannot move '/mnt/lustre/d117i.sanityn/f117i.sanityn' to a subdirectory of itself, '/mnt/lustre/d117i.sanityn/.Trash' mv: cannot move '/mnt/lustre/d117i.sanityn/f117i.sanityn' to a subdirectory of itself, '/mnt/lustre/.Trash' ln: failed to create symbolic link '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument mv: cannot move '/mnt/lustre/d117i.sanityn/d117i.sanityn' to a subdirectory of itself, '/mnt/lustre/d117i.sanityn/.Trash' mv: cannot move '/mnt/lustre/d117i.sanityn/d117i.sanityn' to a subdirectory of itself, '/mnt/lustre/.Trash' PASS 117i (2s) == sanityn test 117j: Move back file from Trash Can ====== 04:34:20 (1766396060) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain tfile.1 -u------------e------- /mnt/lustre/d117j.sanityn/.Trash/tfile.1 --------------e------- /mnt/lustre/d117j.sanityn/tfile.1 List /mnt/lustre/d117j.sanityn/.Trash /mnt/lustre/d117j.sanityn/.Trash: tdir1 /mnt/lustre/d117j.sanityn/.Trash/tdir1: tdir11 tfile12 tfile13 /mnt/lustre/d117j.sanityn/.Trash/tdir1/tdir11: -u-------------------- /mnt/lustre/d117j.sanityn/.Trash/tdir1 /mnt/lustre/d117j.sanityn/.Trash/tdir1: -u-------------------- /mnt/lustre/d117j.sanityn/.Trash/tdir1/tdir11 /mnt/lustre/d117j.sanityn/.Trash/tdir1/tdir11: -u------------e------- /mnt/lustre/d117j.sanityn/.Trash/tdir1/tfile12 -u------------e------- /mnt/lustre/d117j.sanityn/.Trash/tdir1/tfile13 === DONE === mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117j (6s) == sanityn test 117k: Unrm file from Trash Can =========== 04:34:26 (1766396066) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain ==== DONE ==== mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117k (5s) == sanityn test 117l: Clean up files in Trash Can via command 'lfs trash clean' ========================================================== 04:34:31 (1766396071) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Phase 0/PR: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc06:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe7:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbcc:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbf3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 0/PR: remove dir 1:0 from Trash Can Phase 2/PO: remove dir 1:0 from Trash Can Phase 1/EN: remove dir 1:0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc0c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 0/PR: remove dir 1:500 from Trash Can Phase 2/PO: remove dir 1:500 from Trash Can Phase 1/EN: remove dir 1:500 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 2:tdir1 from Trash Can Phase 0/PR: remove dir 3:tdir11 from Trash Can Phase 2/PO: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove file 3:tfile12 from Trash Can Phase 1/EN: remove file 3:tfile13 from Trash Can Phase 2/PO: remove dir 2:tdir1 from Trash Can Phase 1/EN: remove dir 2:tdir1 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc12:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbd2:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbec:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc06:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe7:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbcc:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbf3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Phase 1/EN: remove file 2:tfile22 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc1c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Phase 0/PR: remove dir 1:0 from Trash Can Phase 2/PO: remove dir 1:0 from Trash Can Phase 1/EN: remove dir 1:0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc0c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 0/PR: remove dir 1:500 from Trash Can Phase 2/PO: remove dir 1:500 from Trash Can Phase 1/EN: remove dir 1:500 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 2:tdir1 from Trash Can Phase 0/PR: remove dir 3:tdir11 from Trash Can Phase 2/PO: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove file 3:tfile12 from Trash Can Phase 1/EN: remove file 3:tfile13 from Trash Can Phase 2/PO: remove dir 2:tdir1 from Trash Can Phase 1/EN: remove dir 2:tdir1 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc12:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbd2:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbec:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117l (5s) == sanityn test 117m: Empty Trash Can via the command 'lfs trash clean' ========================================================== 04:34:36 (1766396076) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: tdir1 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0/tdir1: tdir11 tfile12 tfile13 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0/tdir1/tdir11: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: tfile22 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: Phase 0/PR: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc06:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe7:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbcc:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbf3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc1c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Phase 0/PR: remove dir 1:0 from Trash Can Phase 2/PO: remove dir 1:0 from Trash Can Phase 1/EN: remove dir 1:0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc1f:0x0 from Trash Can Phase 0/PR: remove dir 2:tdir1 from Trash Can Phase 0/PR: remove dir 3:tdir11 from Trash Can Phase 2/PO: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove file 3:tfile12 from Trash Can Phase 1/EN: remove file 3:tfile13 from Trash Can Phase 2/PO: remove dir 2:tdir1 from Trash Can Phase 1/EN: remove dir 2:tdir1 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc1f:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc1f:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc1f:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc0c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 0/PR: remove dir 1:500 from Trash Can Phase 2/PO: remove dir 1:500 from Trash Can Phase 1/EN: remove dir 1:500 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc25:0x0 from Trash Can Phase 1/EN: remove file 2:tfile22 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc25:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc25:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc25:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc12:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbd2:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbec:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117m (5s) == sanityn test 117n: Update LinkEA accordingly when moving file into Trash Can ========================================================== 04:34:41 (1766396081) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x829:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x828:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x82a:0x0], name 'tfile22' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x82a:0x0], name 'tfile22' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x82a:0x0], name 'tfile22' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x82a:0x0], name 'tfile22' mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117n (5s) == sanityn test 117o: Set/Clear NOTRASH flag on file ===== 04:34:46 (1766396086) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Set flags 0X80000/0X80040 OK: rc=0 file: /mnt/lustre/d117o.sanityn/tfile.1, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tfile.1, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1/tdir11, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1/tdir11, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1/tfile12, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 lsattr /mnt/lustre/d117o.sanityn/tdir1: ------d--------------- /mnt/lustre/d117o.sanityn/tdir1/tdir11 ------d-------e------- /mnt/lustre/d117o.sanityn/tdir1/tfile12 lsattr /mnt/lustre/d117o.sanityn/tdir1/tdir11: lsattr /mnt/lustre/d117o.sanityn/tdir1/tfile12: ------d-------e------- /mnt/lustre/d117o.sanityn/tdir1/tfile12 file: /mnt/lustre/d117o.sanityn/tdir1/tdir11, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1/tfile12, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 file: /mnt/lustre/d117o.sanityn/tdir1/tfile13, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 Set flags 0X80040/0X80000 OK: rc=0 chattr 1.47.3-wc2 (11-Nov-2025) Flags of /mnt/lustre/d117o.sanityn/tdir1 set as ---------------------- file: /mnt/lustre/d117o.sanityn/tdir1/tfile13, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396088 tfile13 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117o (5s) == sanityn test 117p: Trash Can handling for remote dir/file ========================================================== 04:34:51 (1766396091) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/d117p.sanityn/d117p.sanityn.1/.Trash/f117p.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6532 0x1984 0x280000401 /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: f117p.sanityn /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: d117p.sanityn.01 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0/d117p.sanityn.01: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: f117p.sanityn mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117p (5s) == sanityn test 117pb: Trash Can handling for remote dir/file with multiple levels ========================================================== 04:34:56 (1766396096) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: d117pb.sanityn.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0/d117pb.sanityn.0: d117pb.sanityn.01 f117pb.sanityn.02 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0/d117pb.sanityn.0/d117pb.sanityn.01: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117pb (4s) == sanityn test 117q: Access Trash Can via '.Trash' for striped directory ========================================================== 04:35:00 (1766396100) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain #0 -> 0x200000bd0:0x82d:0x0 #1 -> 0x240000404:0x788:0x0 file.1 file.2 Creating 100 files under striped directory: /mnt/lustre/d117q.sanityn/striped_dir total: 100 open/close in 0.35 seconds: 288.59 ops/second f117q.sanityn0 -> 0 f117q.sanityn1 -> 0 f117q.sanityn2 -> 0 f117q.sanityn3 -> 0 f117q.sanityn4 -> 0 f117q.sanityn5 -> 0 f117q.sanityn6 -> 0 f117q.sanityn7 -> 0 f117q.sanityn8 -> 0 f117q.sanityn9 -> 0 f117q.sanityn10 -> 0 f117q.sanityn11 -> 0 f117q.sanityn12 -> 0 f117q.sanityn13 -> 0 f117q.sanityn14 -> 1 f117q.sanityn15 -> 1 f117q.sanityn16 -> 0 f117q.sanityn17 -> 1 f117q.sanityn18 -> 0 f117q.sanityn19 -> 1 f117q.sanityn20 -> 1 f117q.sanityn21 -> 1 f117q.sanityn22 -> 1 f117q.sanityn23 -> 1 f117q.sanityn24 -> 1 f117q.sanityn25 -> 0 f117q.sanityn26 -> 1 f117q.sanityn27 -> 0 f117q.sanityn28 -> 1 f117q.sanityn29 -> 1 f117q.sanityn30 -> 1 f117q.sanityn31 -> 1 f117q.sanityn32 -> 0 f117q.sanityn33 -> 1 f117q.sanityn34 -> 0 f117q.sanityn35 -> 0 f117q.sanityn36 -> 1 f117q.sanityn37 -> 0 f117q.sanityn38 -> 1 f117q.sanityn39 -> 0 f117q.sanityn40 -> 1 f117q.sanityn41 -> 0 f117q.sanityn42 -> 1 f117q.sanityn43 -> 1 f117q.sanityn44 -> 0 f117q.sanityn45 -> 0 f117q.sanityn46 -> 0 f117q.sanityn47 -> 1 f117q.sanityn48 -> 1 f117q.sanityn49 -> 0 f117q.sanityn50 -> 0 f117q.sanityn51 -> 0 f117q.sanityn52 -> 1 f117q.sanityn53 -> 1 f117q.sanityn54 -> 1 f117q.sanityn55 -> 0 f117q.sanityn56 -> 1 f117q.sanityn57 -> 1 f117q.sanityn58 -> 1 f117q.sanityn59 -> 0 f117q.sanityn60 -> 1 f117q.sanityn61 -> 1 f117q.sanityn62 -> 1 f117q.sanityn63 -> 1 f117q.sanityn64 -> 1 f117q.sanityn65 -> 0 f117q.sanityn66 -> 1 f117q.sanityn67 -> 1 f117q.sanityn68 -> 0 f117q.sanityn69 -> 0 f117q.sanityn70 -> 0 f117q.sanityn71 -> 0 f117q.sanityn72 -> 0 f117q.sanityn73 -> 0 f117q.sanityn74 -> 0 f117q.sanityn75 -> 0 f117q.sanityn76 -> 1 f117q.sanityn77 -> 0 f117q.sanityn78 -> 1 f117q.sanityn79 -> 0 f117q.sanityn80 -> 1 f117q.sanityn81 -> 1 f117q.sanityn82 -> 0 f117q.sanityn83 -> 0 f117q.sanityn84 -> 1 f117q.sanityn85 -> 0 f117q.sanityn86 -> 1 f117q.sanityn87 -> 1 f117q.sanityn88 -> 1 f117q.sanityn89 -> 0 f117q.sanityn90 -> 1 f117q.sanityn91 -> 0 f117q.sanityn92 -> 0 f117q.sanityn93 -> 1 f117q.sanityn94 -> 0 f117q.sanityn95 -> 1 f117q.sanityn96 -> 0 f117q.sanityn97 -> 1 f117q.sanityn98 -> 0 f117q.sanityn99 -> 0 - unlinked 0 (time 1766396104 ; total 0 ; last 0) total: 100 unlinks in 1 seconds: 100.000000 unlinks/second f117q.sanityn0 f117q.sanityn1 f117q.sanityn10 f117q.sanityn11 f117q.sanityn12 f117q.sanityn13 f117q.sanityn14 f117q.sanityn15 f117q.sanityn16 f117q.sanityn17 f117q.sanityn18 f117q.sanityn19 f117q.sanityn2 f117q.sanityn20 f117q.sanityn21 f117q.sanityn22 f117q.sanityn23 f117q.sanityn24 f117q.sanityn25 f117q.sanityn26 f117q.sanityn27 f117q.sanityn28 f117q.sanityn29 f117q.sanityn3 f117q.sanityn30 f117q.sanityn31 f117q.sanityn32 f117q.sanityn33 f117q.sanityn34 f117q.sanityn35 f117q.sanityn36 f117q.sanityn37 f117q.sanityn38 f117q.sanityn39 f117q.sanityn4 f117q.sanityn40 f117q.sanityn41 f117q.sanityn42 f117q.sanityn43 f117q.sanityn44 f117q.sanityn45 f117q.sanityn46 f117q.sanityn47 f117q.sanityn48 f117q.sanityn49 f117q.sanityn5 f117q.sanityn50 f117q.sanityn51 f117q.sanityn52 f117q.sanityn53 f117q.sanityn54 f117q.sanityn55 f117q.sanityn56 f117q.sanityn57 f117q.sanityn58 f117q.sanityn59 f117q.sanityn6 f117q.sanityn60 f117q.sanityn61 f117q.sanityn62 f117q.sanityn63 f117q.sanityn64 f117q.sanityn65 f117q.sanityn66 f117q.sanityn67 f117q.sanityn68 f117q.sanityn69 f117q.sanityn7 f117q.sanityn70 f117q.sanityn71 f117q.sanityn72 f117q.sanityn73 f117q.sanityn74 f117q.sanityn75 f117q.sanityn76 f117q.sanityn77 f117q.sanityn78 f117q.sanityn79 f117q.sanityn8 f117q.sanityn80 f117q.sanityn81 f117q.sanityn82 f117q.sanityn83 f117q.sanityn84 f117q.sanityn85 f117q.sanityn86 f117q.sanityn87 f117q.sanityn88 f117q.sanityn89 f117q.sanityn9 f117q.sanityn90 f117q.sanityn91 f117q.sanityn92 f117q.sanityn93 f117q.sanityn94 f117q.sanityn95 f117q.sanityn96 f117q.sanityn97 f117q.sanityn98 f117q.sanityn99 f117q.sanityn10 f117q.sanityn11 f117q.sanityn12 f117q.sanityn13 f117q.sanityn14 f117q.sanityn15 f117q.sanityn16 f117q.sanityn17 f117q.sanityn18 f117q.sanityn19 f117q.sanityn20 f117q.sanityn21 f117q.sanityn22 f117q.sanityn23 f117q.sanityn24 f117q.sanityn25 f117q.sanityn26 f117q.sanityn27 f117q.sanityn28 f117q.sanityn29 f117q.sanityn30 f117q.sanityn31 f117q.sanityn32 f117q.sanityn33 f117q.sanityn34 f117q.sanityn35 f117q.sanityn36 f117q.sanityn37 f117q.sanityn38 f117q.sanityn39 f117q.sanityn40 f117q.sanityn41 f117q.sanityn42 f117q.sanityn43 f117q.sanityn44 f117q.sanityn45 f117q.sanityn46 f117q.sanityn47 f117q.sanityn48 f117q.sanityn49 f117q.sanityn50 f117q.sanityn51 f117q.sanityn52 f117q.sanityn53 f117q.sanityn54 f117q.sanityn55 f117q.sanityn56 f117q.sanityn57 f117q.sanityn58 f117q.sanityn59 f117q.sanityn60 f117q.sanityn61 f117q.sanityn62 f117q.sanityn63 f117q.sanityn64 f117q.sanityn65 f117q.sanityn66 f117q.sanityn67 f117q.sanityn68 f117q.sanityn69 f117q.sanityn70 f117q.sanityn71 f117q.sanityn72 f117q.sanityn73 f117q.sanityn74 f117q.sanityn75 f117q.sanityn76 f117q.sanityn77 f117q.sanityn78 f117q.sanityn79 f117q.sanityn80 f117q.sanityn81 f117q.sanityn82 f117q.sanityn83 f117q.sanityn84 f117q.sanityn85 f117q.sanityn86 f117q.sanityn87 f117q.sanityn88 f117q.sanityn89 f117q.sanityn90 f117q.sanityn91 f117q.sanityn92 f117q.sanityn93 f117q.sanityn94 f117q.sanityn95 f117q.sanityn96 f117q.sanityn97 f117q.sanityn98 f117q.sanityn99 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117q (10s) == sanityn test 117r: Verify the deleting a striped directory on Trash Can ========================================================== 04:35:10 (1766396110) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain #0 -> 0x200000bd0:0x82f:0x0 #1 -> 0x240000404:0x789:0x0 Creating 50 files under striped directory: /mnt/lustre/d117r.sanityn/striped_dir total: 50 open/close in 0.13 seconds: 378.44 ops/second f117r.sanityn0 -> 0 f117r.sanityn1 -> 0 f117r.sanityn2 -> 0 f117r.sanityn3 -> 0 f117r.sanityn4 -> 0 f117r.sanityn5 -> 0 f117r.sanityn6 -> 0 f117r.sanityn7 -> 0 f117r.sanityn8 -> 0 f117r.sanityn9 -> 0 f117r.sanityn10 -> 1 f117r.sanityn11 -> 1 f117r.sanityn12 -> 1 f117r.sanityn13 -> 1 f117r.sanityn14 -> 1 f117r.sanityn15 -> 0 f117r.sanityn16 -> 1 f117r.sanityn17 -> 1 f117r.sanityn18 -> 0 f117r.sanityn19 -> 1 f117r.sanityn20 -> 0 f117r.sanityn21 -> 0 f117r.sanityn22 -> 0 f117r.sanityn23 -> 0 f117r.sanityn24 -> 0 f117r.sanityn25 -> 1 f117r.sanityn26 -> 0 f117r.sanityn27 -> 1 f117r.sanityn28 -> 1 f117r.sanityn29 -> 1 f117r.sanityn30 -> 1 f117r.sanityn31 -> 0 f117r.sanityn32 -> 0 f117r.sanityn33 -> 0 f117r.sanityn34 -> 1 f117r.sanityn35 -> 0 f117r.sanityn36 -> 0 f117r.sanityn37 -> 0 f117r.sanityn38 -> 0 f117r.sanityn39 -> 1 f117r.sanityn40 -> 0 f117r.sanityn41 -> 1 f117r.sanityn42 -> 0 f117r.sanityn43 -> 1 f117r.sanityn44 -> 0 f117r.sanityn45 -> 1 f117r.sanityn46 -> 0 f117r.sanityn47 -> 0 f117r.sanityn48 -> 1 f117r.sanityn49 -> 0 - unlinked 0 (time 1766396112 ; total 0 ; last 0) total: 50 unlinks in 0 seconds: inf unlinks/second Check files in Trash Can: f117r.sanityn0 f117r.sanityn1 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn2 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn3 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn4 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 f117r.sanityn5 f117r.sanityn6 f117r.sanityn7 f117r.sanityn8 f117r.sanityn9 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 FID for /mnt/lustre/d117r.sanityn/striped_dir [0x200000bd1:0xc72:0x0] pfid[0]=0x200000bd0:0x82f:0x0 [0x200000400:0x2:0x0] [0x200000bd0:0x830:0x0] pfid[1]=0x240000404:0x789:0x0 [0x240000400:0x2:0x0] [0x240000400:0xeda:0x0] Obtain LMVEA for virtual dir /mnt/lustre/d117r.sanityn/striped_dir/.Trash: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x830:0x0] 1 [0x240000400:0xeda:0x0] #0 -> name: 0x200000bd0:0x82f:0x0 fid: 0x200000bd0:0x830:0x0 #1 -> name: 0x240000404:0x789:0x0 fid: 0x240000400:0xeda:0x0 /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: striped_dir /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0/striped_dir: f117r.sanityn0 f117r.sanityn1 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn2 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn3 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn4 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 f117r.sanityn5 f117r.sanityn6 f117r.sanityn7 f117r.sanityn8 f117r.sanityn9 file.1 file.2 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: Obtain LMVEA for striped dir /mnt/lustre/d117r.sanityn/striped_dir: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x830:0x0] 1 [0x240000400:0xeda:0x0] [0x200000bd0:0x832:0x0] #0 -> name: 0x200000bd0:0x82f:0x0 fid: 0x200000bd0:0x830:0x0 #1 -> name: 0x240000404:0x789:0x0 fid: 0x240000400:0xeda:0x0 lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x830:0x0] 1 [0x240000400:0xeda:0x0] f117r.sanityn0 f117r.sanityn1 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn2 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn3 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn4 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 f117r.sanityn5 f117r.sanityn6 f117r.sanityn7 f117r.sanityn8 f117r.sanityn9 file.1 file.2 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn0, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn1, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn10, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn11, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn12, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn13, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn14, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn15, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn16, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn17, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn18, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn19, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn2, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn20, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn21, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn22, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn23, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn24, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn25, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn26, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn27, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn28, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn29, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn3, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn30, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn31, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn32, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn33, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn34, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn35, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn36, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn37, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn38, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn39, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn4, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn40, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn41, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn42, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn43, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn44, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn45, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn46, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn47, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn48, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn49, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn5, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn6, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn7, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn8, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn9, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/file.1, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 file: /mnt/lustre/d117r.sanityn/striped_dir/file.2, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396112 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117r (7s) == sanityn test 117s: Create trash shard if it does not exist ========================================================== 04:35:17 (1766396117) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: file.1 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x835:0x0] 1 [0x240000404:0x78a:0x0] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117s (6s) == sanityn test 117t: UID/GID/ProjID for files in Trash Can ========================================================== 04:35:23 (1766396123) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain file: /mnt/lustre/d117t.sanityn/.Trash/tfile1, flags: 0X80002 unrm, uid: 0 gid: 0 projid: 0 timestamp: 1766396125 File: /mnt/lustre/d117t.sanityn/.Trash/tfile1 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826937494 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:35:25.000000000 -0500 Modify: 2025-12-22 04:35:25.000000000 -0500 Change: 2025-12-22 04:35:25.000000000 -0500 Birth: 2025-12-22 04:35:25.000000000 -0500 trash_can_uid=4294967294 trash_can_gid=4294967294 trash_can_projid=4294967294 === Checking ugp ID on MDT === oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519320 Type: regular Mode: 0644 Flags: 0x80002 Generation: 1963963401 Version: 0x00000002:000191c6 User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x694910dd:00000000 -- Mon Dec 22 04:35:25 2025 atime: 0x694910dd:00000000 -- Mon Dec 22 04:35:25 2025 mtime: 0x694910dd:00000000 -- Mon Dec 22 04:35:25 2025 crtime: 0x694910dd:564eba00 -- Mon Dec 22 04:35:25 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200000bd1:0xc96:0x0] compat=0 incompat=0 user.job (5) = "lfs.0" linkea: idx=0 parent=[0x200000bd0:0x839:0x0] name='tfile1' trusted.lov (56) = d0 0b d1 0b 01 00 00 00 96 0c 00 00 00 00 00 00 d1 0b 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 b0 19 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.unrm (24) = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 dd 10 49 69 00 00 00 00 EXTENTS: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) === Checking ugp ID on OST === oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 3503 Type: regular Mode: 0666 Flags: 0x80000 Generation: 2629152413 Version: 0x00000000:00000000 User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x00000000:00000000 -- Wed Dec 31 19:00:00 1969 atime: 0x00000000:00000000 -- Wed Dec 31 19:00:00 1969 mtime: 0x00000000:00000000 -- Wed Dec 31 19:00:00 1969 crtime: 0x694910c7:61094f00 -- Mon Dec 22 04:35:03 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x280000401:0x19b0:0x0] compat=8 incompat=0 EXTENTS: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117t.sanityn/tfile2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117t.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [unlink] [/mnt/lustre/d117t.sanityn/tfile2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [rmdir] [/mnt/lustre/d117t.sanityn/tdir2] file: /mnt/lustre/d117t.sanityn/.Trash/tfile2, flags: 0X80002 unrm, uid: 500 gid: 500 projid: 0 timestamp: 1766396129 File: /mnt/lustre/d117t.sanityn/.Trash/tfile2 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826937495 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 500/sanityusr) Gid: ( 500/sanityusr) Access: 2025-12-22 04:35:29.000000000 -0500 Modify: 2025-12-22 04:35:29.000000000 -0500 Change: 2025-12-22 04:35:29.000000000 -0500 Birth: 2025-12-22 04:35:29.000000000 -0500 using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre opts: Stopping clients: oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg218-server tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg218-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg218-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg218-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/58763/testresults/sanityn-ldiskfs-DNE-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg218-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0000 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0001 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Started clients oleg218-client.virtnet: 192.168.202.118@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Started clients oleg218-client.virtnet: 192.168.202.118@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff92ff04cbe800.idle_timeout=debug osc.lustre-OST0000-osc-ffff92ff190b4800.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff04cbe800.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff190b4800.idle_timeout=debug disable quota as required mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117t (113s) == sanityn test 117u: statfs() call should not account the space in Trash Can ========================================================== 04:37:16 (1766396236) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_projid=1000 mdd.lustre-MDT0001.trash_can_projid=1000 llite.lustre-ffff92ff04cbe800.trash_can_projid=1000 llite.lustre-ffff92ff190b4800.trash_can_projid=1000 total=7666232 used=86852 avail=7127188 free=7579380 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.118@tcp:/lustre 7666232 86852 7127188 2% /mnt/lustre 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.737118 s, 142 MB/s new total=7666232 used=0 avail=7666232 free=7666232 Filesystem 1024-blocks Used Available Capacity Mounted on 192.168.202.118@tcp:/lustre 7666232 0 7666232 0% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 5392 1265912 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3120 1268184 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 131128 3475892 4% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 58128 3548892 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 189256 7024784 3% /mnt/lustre /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: f117u.sanityn /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: PROJID: 1000 Quota Info: Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 102404 0 0 - 1 0 0 - lustre-MDT0000 0 - 0 - 1 - 0 - lustre-MDT0001 0 - 0 - 0 - 0 - lustre-OST0000 102404 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for usr green (uid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-MDT0001 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 total=7666232 used=189256 avail=7024784 free=7476976 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 5392 1265912 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3120 1268184 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 131128 3475892 4% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 58128 3548892 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 189256 7024784 3% /mnt/lustre new total=7666232 used=58128 avail=7382008 free=7608104 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 5392 1265912 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3120 1268184 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 0 3833116 0% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 58128 3548892 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 58128 7382008 1% /mnt/lustre llite.lustre-ffff92ff04cbe800.trash_can_projid=4294967294 llite.lustre-ffff92ff190b4800.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_uid=4294967294 mdd.lustre-MDT0001.trash_can_uid=4294967294 mdd.lustre-MDT0000.trash_can_gid=4294967294 mdd.lustre-MDT0001.trash_can_gid=4294967294 mdd.lustre-MDT0000.trash_can_projid=4294967294 mdd.lustre-MDT0001.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117u (11s) == sanityn test 117ub: statfs() account the inode usage in Trash Can ========================================================== 04:37:27 (1766396247) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 12493 1011507 2% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1024000 9370 1014630 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 262144 3683 258461 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 262144 3678 258466 2% /mnt/lustre[OST:1] filesystem_summary: 538790 21863 516927 5% /mnt/lustre total: 1000 open/close in 2.20 seconds: 455.19 ops/second - unlinked 0 (time 1766396254 ; total 0 ; last 0) total: 1000 unlinks in 1 seconds: 1000.000000 unlinks/second === w/o Trash Can account === lustre-MDT0000_UUID 1024000 13495 1010505 2% /mnt/lustre[MDT:0] === w/ Trash Can account === lustre-MDT0000_UUID 1024000 12495 1011505 2% /mnt/lustre[MDT:0] tcu_ffree: 1011505 ffree: 1010505 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117ub (16s) == sanityn test 117v: Test for Trash Can UID/GID/PROJID setting ========================================================== 04:37:43 (1766396263) mdd.lustre-MDT0000.trash_can_uid=10 mdd.lustre-MDT0001.trash_can_uid=10 mdd.lustre-MDT0000.trash_can_gid=10 mdd.lustre-MDT0001.trash_can_gid=10 mdd.lustre-MDT0000.trash_can_projid=10 mdd.lustre-MDT0001.trash_can_projid=10 llite.lustre-ffff92ff04cbe800.trash_can_projid=10 llite.lustre-ffff92ff190b4800.trash_can_projid=10 mdd.lustre-MDT0000.trash_can_uid=-1 mdd.lustre-MDT0001.trash_can_uid=-1 mdd.lustre-MDT0000.trash_can_gid=-1 mdd.lustre-MDT0001.trash_can_gid=-1 mdd.lustre-MDT0000.trash_can_projid=-1 mdd.lustre-MDT0001.trash_can_projid=-1 llite.lustre-ffff92ff04cbe800.trash_can_projid=-1 llite.lustre-ffff92ff190b4800.trash_can_projid=-1 mdd.lustre-MDT0000.trash_can_uid=-2 mdd.lustre-MDT0001.trash_can_uid=-2 mdd.lustre-MDT0000.trash_can_gid=-2 mdd.lustre-MDT0001.trash_can_gid=-2 mdd.lustre-MDT0000.trash_can_projid=-2 mdd.lustre-MDT0001.trash_can_projid=-2 llite.lustre-ffff92ff04cbe800.trash_can_projid=-2 llite.lustre-ffff92ff190b4800.trash_can_projid=-2 mdd.lustre-MDT0000.trash_can_uid=original mdd.lustre-MDT0001.trash_can_uid=original mdd.lustre-MDT0000.trash_can_gid=original mdd.lustre-MDT0001.trash_can_gid=original mdd.lustre-MDT0000.trash_can_projid=original mdd.lustre-MDT0001.trash_can_projid=original llite.lustre-ffff92ff04cbe800.trash_can_projid=original llite.lustre-ffff92ff190b4800.trash_can_projid=original mdd.lustre-MDT0000.trash_can_uid=default mdd.lustre-MDT0001.trash_can_uid=default mdd.lustre-MDT0000.trash_can_gid=default mdd.lustre-MDT0001.trash_can_gid=default mdd.lustre-MDT0000.trash_can_projid=default mdd.lustre-MDT0001.trash_can_projid=default llite.lustre-ffff92ff04cbe800.trash_can_projid=default llite.lustre-ffff92ff190b4800.trash_can_projid=default llite.lustre-ffff92ff04cbe800.trash_can_projid=4294967294 llite.lustre-ffff92ff190b4800.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_uid=4294967294 mdd.lustre-MDT0001.trash_can_uid=4294967294 mdd.lustre-MDT0000.trash_can_gid=4294967294 mdd.lustre-MDT0001.trash_can_gid=4294967294 mdd.lustre-MDT0000.trash_can_projid=4294967294 mdd.lustre-MDT0001.trash_can_projid=4294967294 PASS 117v (8s) == sanityn test 117w: Test Trash Can with original UID/GID/PROJID ========================================================== 04:37:51 (1766396271) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_uid=original mdd.lustre-MDT0001.trash_can_uid=original mdd.lustre-MDT0000.trash_can_gid=original mdd.lustre-MDT0001.trash_can_gid=original mdd.lustre-MDT0000.trash_can_projid=original mdd.lustre-MDT0001.trash_can_projid=original 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.660783 s, 159 MB/s /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x5:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: tdir.1 tfile.1 /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0/tdir.1: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519327 Type: regular Mode: 0644 Flags: 0x80002 Generation: 3706749119 Version: 0x00000003:000010bb User: 0 Group: 0 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491173:00000000 -- Mon Dec 22 04:37:55 2025 atime: 0x69491172:00000000 -- Mon Dec 22 04:37:54 2025 mtime: 0x69491173:00000000 -- Mon Dec 22 04:37:55 2025 crtime: 0x69491172:9e124f00 -- Mon Dec 22 04:37:54 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3f0:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 f0 03 00 00 00 00 00 00 a1 13 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 c0 02 00 00 00 b7 1b 00 00 00 00 00 00 00 00 00 00 01 00 00 00 user.job (4) = "dd.0" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tfile.1' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 40 06 00 00 00 00 08 20 03 00 00 00 00 00 EXTENTS: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519326 Type: directory Mode: 0755 Flags: 0x2 Generation: 2464958776 Version: 0x00000003:000010ba User: 0 Group: 0 Project: 0 Size: 4096 File ACL: 0 Links: 4 Blockcount: 8 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491173:00000000 -- Mon Dec 22 04:37:55 2025 atime: 0x69491172:00000000 -- Mon Dec 22 04:37:54 2025 mtime: 0x69491172:00000000 -- Mon Dec 22 04:37:54 2025 crtime: 0x69491172:9ca41900 -- Mon Dec 22 04:37:54 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3ef:0x0] compat=0 incompat=0 user.job (7) = "mkdir.0" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tdir.1' BLOCKS: (0):345721 TOTAL: 1 oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117w.sanityn/tdir.2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117w.sanityn/tfile.2] oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519330 Type: regular Mode: 0644 Flags: 0x80002 Generation: 3929632453 Version: 0x00000003:000010f3 User: 500 Group: 500 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491176:00000000 -- Mon Dec 22 04:37:58 2025 atime: 0x69491176:00000000 -- Mon Dec 22 04:37:58 2025 mtime: 0x69491176:00000000 -- Mon Dec 22 04:37:58 2025 crtime: 0x69491175:bbdfb400 -- Mon Dec 22 04:37:57 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3f2:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 f2 03 00 00 00 00 00 00 a1 13 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 b7 1b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 user.job (9) = "touch.500" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tfile.2' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EXTENTS: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519329 Type: directory Mode: 0755 Flags: 0x2 Generation: 3070388339 Version: 0x00000003:000010f2 User: 500 Group: 500 Project: 0 Size: 4096 File ACL: 0 Links: 4 Blockcount: 8 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491176:00000000 -- Mon Dec 22 04:37:58 2025 atime: 0x69491176:00000000 -- Mon Dec 22 04:37:58 2025 mtime: 0x69491176:00000000 -- Mon Dec 22 04:37:58 2025 crtime: 0x69491175:ba717e00 -- Mon Dec 22 04:37:57 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3f1:0x0] compat=0 incompat=0 user.job (9) = "mkdir.500" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tdir.2' BLOCKS: (0):345723 TOTAL: 1 oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) mdd.lustre-MDT0000.trash_can_uid=4294967294 mdd.lustre-MDT0001.trash_can_uid=4294967294 mdd.lustre-MDT0000.trash_can_gid=4294967294 mdd.lustre-MDT0001.trash_can_gid=4294967294 mdd.lustre-MDT0000.trash_can_projid=4294967294 mdd.lustre-MDT0001.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117w (11s) == sanityn test 117x: Unrm Files from Trash Can should also restore UID/GID/PROJID ========================================================== 04:38:02 (1766396282) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tdir1 trusted.unrm=0sAAAAAAAAAAAAAAAAAAAAAHwRSWkAAAAA getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tfile1 trusted.unrm=0sAAAAAAAAAAAAAAAAAAAAAHwRSWkAAAAA /mnt/lustre/d117x.sanityn/tdir1: trusted.unrm: No such attribute /mnt/lustre/d117x.sanityn/tfile1: trusted.unrm: No such attribute running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117x.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117x.sanityn/tfile2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [rmdir] [/mnt/lustre/d117x.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [unlink] [/mnt/lustre/d117x.sanityn/tfile2] getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tdir2 trusted.unrm=0sAAAAAPQBAAD0AQAAAAAAAHwRSWkAAAAA getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tfile2 trusted.unrm=0sAAAAAPQBAAD0AQAAAAAAAHwRSWkAAAAA /mnt/lustre/d117x.sanityn/tdir2: trusted.unrm: No such attribute /mnt/lustre/d117x.sanityn/tfile2: trusted.unrm: No such attribute mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117x (5s) == sanityn test 117ya: Test with symbol links with Trash Can ========================================================== 04:38:07 (1766396287) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain total 8 lrwxrwxrwx 1 root root 32 Dec 22 04:38 lnkd5 -> /mnt/lustre/d117ya.sanityn/tdir1 lrwxrwxrwx 1 root root 33 Dec 22 04:38 lnkf4 -> /mnt/lustre/d117ya.sanityn/tfile3 drwxr-xr-x 3 root root 4096 Dec 22 04:38 tdir1 drwxr-xr-x 2 root root 4096 Dec 22 04:38 tdir2 -rw-r--r-- 1 root root 0 Dec 22 04:38 tfile3 After unlink /mnt/lustre/d117ya.sanityn/lnkf4 and /mnt/lustre/d117ya.sanityn/lnkd5 /mnt/lustre/d117ya.sanityn/.Trash: total 4 lrwxrwxrwx 1 root root 32 Dec 22 04:38 lnkd5 -> /mnt/lustre/d117ya.sanityn/tdir1 lrwxrwxrwx 1 root root 33 Dec 22 04:38 lnkf4 -> /mnt/lustre/d117ya.sanityn/tfile3 d--------- 3 root root 4096 Dec 22 04:38 tdir1 -rw-r--r-- 1 root root 0 Dec 22 04:38 tfile3 /mnt/lustre/d117ya.sanityn/.Trash/tdir1: total 4 drwxr-xr-x 4 root root 4096 Dec 22 04:38 tdir13 -rw-r--r-- 1 root root 0 Dec 22 04:38 tfile11 -rw-r--r-- 1 root root 0 Dec 22 04:38 tfile12 /mnt/lustre/d117ya.sanityn/.Trash/tdir1/tdir13: total 0 file: /mnt/lustre/d117ya.sanityn/.Trash/lnkf4, flags: 0X2 unrm, uid: 0 gid: 0 projid: 0 timestamp: 1766396289 file: /mnt/lustre/d117ya.sanityn/.Trash/lnkd5, flags: 0X2 unrm, uid: 0 gid: 0 projid: 0 timestamp: 1766396289 Unrm stub file 'lnkf4' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/lnkf4' is in Trash Can Path '/mnt/lustre/d117ya.sanityn/.Trash/lnkf4' is a symlink Handle symlink /mnt/lustre/d117ya.sanityn/.Trash/lnkf4 Unrm stub file 'tfile3' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/tfile3' is in Trash Can Set flags 0X80002/0X80000 OK: rc=0 Unrm stub file 'tdir1' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/tdir1' is in Trash Can Set flags 0X2/0 OK: rc=0 Handle dir 0:'tdir13' under directory '/mnt/lustre/d117ya.sanityn/tdir1' Handle special file 0:/mnt/lustre/d117ya.sanityn/tdir1/tdir13 Handle file 0:'tfile11' under directory '/mnt/lustre/d117ya.sanityn/tdir1' Handle special file 0:/mnt/lustre/d117ya.sanityn/tdir1/tfile11 Handle file 0:'tfile12' under directory '/mnt/lustre/d117ya.sanityn/tdir1' Handle special file 0:/mnt/lustre/d117ya.sanityn/tdir1/tfile12 Unrm stub file 'lnkd5' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/lnkd5' is in Trash Can Path '/mnt/lustre/d117ya.sanityn/.Trash/lnkd5' is a symlink Handle symlink /mnt/lustre/d117ya.sanityn/.Trash/lnkd5 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117ya (4s) == sanityn test 117yb: Compare the directories containing symlinks after unrm recover ========================================================== 04:38:12 (1766396292) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Only in /etc/pam.d: system-auth Only in /etc/postfix: master.cf.proto Only in /etc: subuid Only in /etc: subuid- sanityn test_117yb: @@@@@@ FAIL: /etc and /mnt/lustre/d117yb.sanityn/etc diff Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:8873:test_117yb() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:8875:main() Dumping lctl log to /tmp/testlogs//sanityn.test_117yb.*.1766396322.log rsync: chown "/tmp/testlogs/.sanityn.test_117yb.debug_log.oleg218-server.1766396322.log.DgeVZG" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_117yb.dmesg.oleg218-server.1766396322.log.GS63eb" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid FAIL 117yb (34s) == sanityn test 117yc: Restoring UID/GID/PROJID for symlinks works ========================================================== 04:38:45 (1766396325) using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg218-client.virtnet /mnt/lustre (opts:) Stopping client oleg218-client.virtnet /mnt/lustre opts: Stopping clients: oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg218-server tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg218-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg218-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg218-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/58763/testresults/sanityn-ldiskfs-DNE-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.202.18,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg218-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0000 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0001 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Starting client oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre Started clients oleg218-client.virtnet: 192.168.202.118@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Starting client oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2 Started clients oleg218-client.virtnet: 192.168.202.118@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff92ff2fe2d800.idle_timeout=debug osc.lustre-OST0000-osc-ffff92ff3d48e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff2fe2d800.idle_timeout=debug osc.lustre-OST0001-osc-ffff92ff3d48e000.idle_timeout=debug disable quota as required mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117yc.sanityn/tdir1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117yc.sanityn/tfile1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [ln] [-s] [/mnt/lustre/d117yc.sanityn/tfile1] [/mnt/lustre/d117yc.sanityn/lnkf3] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [ln] [-s] [/mnt/lustre/d117yc.sanityn/tdir1] [/mnt/lustre/d117yc.sanityn/lnkd4] getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117yc.sanityn/.Trash/lnkf3 trusted.unrm=0sAAAAAPQBAAD0AQAAyAAAAOoRSWkAAAAA getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117yc.sanityn/.Trash/lnkd4 trusted.unrm=0sAAAAAPQBAAD0AQAAAAAAAOoRSWkAAAAA Unrm stub file 'lnkf3' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/lnkf3' is in Trash Can Path '/mnt/lustre/d117yc.sanityn/.Trash/lnkf3' is a symlink Handle symlink /mnt/lustre/d117yc.sanityn/.Trash/lnkf3 Unrm stub file 'tdir1' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/tdir1' is in Trash Can Set flags 0X2/0 OK: rc=0 Unrm stub file 'tfile1' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/tfile1' is in Trash Can Set flags 0X80002/0X80000 OK: rc=0 Unrm stub file 'lnkd4' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/lnkd4' is in Trash Can Path '/mnt/lustre/d117yc.sanityn/.Trash/lnkd4' is a symlink Handle symlink /mnt/lustre/d117yc.sanityn/.Trash/lnkd4 /mnt/lustre/d117yc.sanityn/lnkf3: trusted.unrm: No such attribute /mnt/lustre/d117yc.sanityn/lnkd4: trusted.unrm: No such attribute mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117yc (73s) == sanityn test 117za: Test Trash Can for UID Trash Can type with nodemap ========================================================== 04:39:58 (1766396398) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200001b71:0x6:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: On MGS 192.168.202.118, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.202.118, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200001b71:0x6:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0/0x200001b71:0x6:0x0: tfile.1 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117za.sanityn/tfile.2] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200001b71:0x6:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0 500 /mnt/lustre/.lustre/trash/MDT0000/tnm/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0/0x200001b71:0x6:0x0: tfile.1 /mnt/lustre/.lustre/trash/MDT0000/tnm/500: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/500/0x200001b71:0x6:0x0: tfile.2 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 520229 Type: regular Mode: 0644 Flags: 0x80002 Generation: 404729162 Version: 0x00000004:000000fd User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x6949120a:00000000 -- Mon Dec 22 04:40:26 2025 atime: 0x6949120a:00000000 -- Mon Dec 22 04:40:26 2025 mtime: 0x6949120a:00000000 -- Mon Dec 22 04:40:26 2025 crtime: 0x6949120a:3d831200 -- Mon Dec 22 04:40:26 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200001b71:0x9:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 09 00 00 00 00 00 00 00 71 1b 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 c0 02 00 00 00 c3 1c 00 00 00 00 00 00 00 00 00 00 01 00 00 00 user.job (9) = "touch.500" linkea: idx=0 parent=[0x200001b70:0x7:0x0] name='tfile.2' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.unrm (24) = 00 00 00 00 f4 01 00 00 f4 01 00 00 00 00 00 00 0a 12 49 69 00 00 00 00 EXTENTS: file: /mnt/lustre/.lustre/trash/MDT0000/tnm/500/0x200001b71:0x6:0x0/tfile.2, flags: 0X80002 unrm, uid: 500 gid: 500 projid: 0 timestamp: 1766396426 On MGS 192.168.202.118, active = nodemap.active=0 waiting 10 secs for sync mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117za (56s) == sanityn test 117zb: Test Trash Can for PLAIN Trash Can type with nodemap ========================================================== 04:40:54 (1766396454) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 0x200001b71:0xa:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xa:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: On MGS 192.168.202.118, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.202.118, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 0x200001b71:0xa:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xa:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0x200001b71:0xa:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0x200001b71:0xa:0x0: tfile.1 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117zb.sanityn/tfile.2] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x82d:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 0x200001b71:0xa:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x82d:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xa:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0x200001b71:0xa:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0x200001b71:0xa:0x0: tfile.1 tfile.2 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x788:0x0 0x240000404:0x78a:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x788:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x78a:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 520229 Type: regular Mode: 0644 Flags: 0x80002 Generation: 3482690890 Version: 0x00000004:0000021e User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x6949124e:00000000 -- Mon Dec 22 04:41:34 2025 atime: 0x6949124e:00000000 -- Mon Dec 22 04:41:34 2025 mtime: 0x6949124e:00000000 -- Mon Dec 22 04:41:34 2025 crtime: 0x6949124e:82e44d00 -- Mon Dec 22 04:41:34 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200001b71:0xd:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 0d 00 00 00 00 00 00 00 71 1b 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 c5 1c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 user.job (9) = "touch.500" linkea: idx=0 parent=[0x200001b70:0x9:0x0] name='tfile.2' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.unrm (24) = 00 00 00 00 f4 01 00 00 f4 01 00 00 00 00 00 00 4e 12 49 69 00 00 00 00 EXTENTS: file: /mnt/lustre/.lustre/trash/MDT0000/tnm//0x200001b71:0xa:0x0/tfile.2, flags: 0X80002 unrm, uid: 500 gid: 500 projid: 0 timestamp: 1766396494 On MGS 192.168.202.118, active = nodemap.active=0 waiting 10 secs for sync mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117zb (59s) == sanityn test 117zc: TCU: Test striped dir for UID Trash Can type ========================================================== 04:41:53 (1766396513) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid Creating 100 files under striped directory: /mnt/lustre/d117zc.sanityn/striped_dir total: 100 open/close in 0.95 seconds: 104.82 ops/second f117zc.sanityn0 -> 1 f117zc.sanityn1 -> 1 f117zc.sanityn2 -> 1 f117zc.sanityn3 -> 1 f117zc.sanityn4 -> 1 f117zc.sanityn5 -> 1 f117zc.sanityn6 -> 1 f117zc.sanityn7 -> 1 f117zc.sanityn8 -> 1 f117zc.sanityn9 -> 1 f117zc.sanityn10 -> 0 f117zc.sanityn11 -> 0 f117zc.sanityn12 -> 0 f117zc.sanityn13 -> 1 f117zc.sanityn14 -> 0 f117zc.sanityn15 -> 1 f117zc.sanityn16 -> 1 f117zc.sanityn17 -> 0 f117zc.sanityn18 -> 1 f117zc.sanityn19 -> 1 f117zc.sanityn20 -> 0 f117zc.sanityn21 -> 1 f117zc.sanityn22 -> 0 f117zc.sanityn23 -> 0 f117zc.sanityn24 -> 0 f117zc.sanityn25 -> 1 f117zc.sanityn26 -> 0 f117zc.sanityn27 -> 0 f117zc.sanityn28 -> 1 f117zc.sanityn29 -> 1 f117zc.sanityn30 -> 1 f117zc.sanityn31 -> 0 f117zc.sanityn32 -> 0 f117zc.sanityn33 -> 0 f117zc.sanityn34 -> 0 f117zc.sanityn35 -> 1 f117zc.sanityn36 -> 0 f117zc.sanityn37 -> 1 f117zc.sanityn38 -> 1 f117zc.sanityn39 -> 1 f117zc.sanityn40 -> 1 f117zc.sanityn41 -> 1 f117zc.sanityn42 -> 0 f117zc.sanityn43 -> 1 f117zc.sanityn44 -> 0 f117zc.sanityn45 -> 0 f117zc.sanityn46 -> 0 f117zc.sanityn47 -> 0 f117zc.sanityn48 -> 0 f117zc.sanityn49 -> 1 f117zc.sanityn50 -> 0 f117zc.sanityn51 -> 0 f117zc.sanityn52 -> 0 f117zc.sanityn53 -> 1 f117zc.sanityn54 -> 0 f117zc.sanityn55 -> 1 f117zc.sanityn56 -> 0 f117zc.sanityn57 -> 1 f117zc.sanityn58 -> 1 f117zc.sanityn59 -> 0 f117zc.sanityn60 -> 1 f117zc.sanityn61 -> 0 f117zc.sanityn62 -> 0 f117zc.sanityn63 -> 0 f117zc.sanityn64 -> 1 f117zc.sanityn65 -> 0 f117zc.sanityn66 -> 0 f117zc.sanityn67 -> 1 f117zc.sanityn68 -> 1 f117zc.sanityn69 -> 1 f117zc.sanityn70 -> 0 f117zc.sanityn71 -> 0 f117zc.sanityn72 -> 0 f117zc.sanityn73 -> 0 f117zc.sanityn74 -> 1 f117zc.sanityn75 -> 0 f117zc.sanityn76 -> 1 f117zc.sanityn77 -> 0 f117zc.sanityn78 -> 0 f117zc.sanityn79 -> 1 f117zc.sanityn80 -> 0 f117zc.sanityn81 -> 1 f117zc.sanityn82 -> 1 f117zc.sanityn83 -> 1 f117zc.sanityn84 -> 0 f117zc.sanityn85 -> 1 f117zc.sanityn86 -> 1 f117zc.sanityn87 -> 1 f117zc.sanityn88 -> 0 f117zc.sanityn89 -> 1 f117zc.sanityn90 -> 0 f117zc.sanityn91 -> 1 f117zc.sanityn92 -> 1 f117zc.sanityn93 -> 0 f117zc.sanityn94 -> 0 f117zc.sanityn95 -> 0 f117zc.sanityn96 -> 1 f117zc.sanityn97 -> 0 f117zc.sanityn98 -> 0 f117zc.sanityn99 -> 1 #0 -> 0x200001b70:0xa:0x0 #1 -> 0x2400013a0:0x1:0x0 - unlinked 0 (time 1766396520 ; total 0 ; last 0) total: 100 unlinks in 0 seconds: inf unlinks/second f117zc.sanityn0 f117zc.sanityn1 f117zc.sanityn10 f117zc.sanityn11 f117zc.sanityn12 f117zc.sanityn13 f117zc.sanityn14 f117zc.sanityn15 f117zc.sanityn16 f117zc.sanityn17 f117zc.sanityn18 f117zc.sanityn19 f117zc.sanityn2 f117zc.sanityn20 f117zc.sanityn21 f117zc.sanityn22 f117zc.sanityn23 f117zc.sanityn24 f117zc.sanityn25 f117zc.sanityn26 f117zc.sanityn27 f117zc.sanityn28 f117zc.sanityn29 f117zc.sanityn3 f117zc.sanityn30 f117zc.sanityn31 f117zc.sanityn32 f117zc.sanityn33 f117zc.sanityn34 f117zc.sanityn35 f117zc.sanityn36 f117zc.sanityn37 f117zc.sanityn38 f117zc.sanityn39 f117zc.sanityn4 f117zc.sanityn40 f117zc.sanityn41 f117zc.sanityn42 f117zc.sanityn43 f117zc.sanityn44 f117zc.sanityn45 f117zc.sanityn46 f117zc.sanityn47 f117zc.sanityn48 f117zc.sanityn49 f117zc.sanityn5 f117zc.sanityn50 f117zc.sanityn51 f117zc.sanityn52 f117zc.sanityn53 f117zc.sanityn54 f117zc.sanityn55 f117zc.sanityn56 f117zc.sanityn57 f117zc.sanityn58 f117zc.sanityn59 f117zc.sanityn6 f117zc.sanityn60 f117zc.sanityn61 f117zc.sanityn62 f117zc.sanityn63 f117zc.sanityn64 f117zc.sanityn65 f117zc.sanityn66 f117zc.sanityn67 f117zc.sanityn68 f117zc.sanityn69 f117zc.sanityn7 f117zc.sanityn70 f117zc.sanityn71 f117zc.sanityn72 f117zc.sanityn73 f117zc.sanityn74 f117zc.sanityn75 f117zc.sanityn76 f117zc.sanityn77 f117zc.sanityn78 f117zc.sanityn79 f117zc.sanityn8 f117zc.sanityn80 f117zc.sanityn81 f117zc.sanityn82 f117zc.sanityn83 f117zc.sanityn84 f117zc.sanityn85 f117zc.sanityn86 f117zc.sanityn87 f117zc.sanityn88 f117zc.sanityn89 f117zc.sanityn9 f117zc.sanityn90 f117zc.sanityn91 f117zc.sanityn92 f117zc.sanityn93 f117zc.sanityn94 f117zc.sanityn95 f117zc.sanityn96 f117zc.sanityn97 f117zc.sanityn98 f117zc.sanityn99 f117zc.sanityn10 f117zc.sanityn11 f117zc.sanityn12 f117zc.sanityn13 f117zc.sanityn14 f117zc.sanityn15 f117zc.sanityn16 f117zc.sanityn17 f117zc.sanityn18 f117zc.sanityn19 f117zc.sanityn20 f117zc.sanityn21 f117zc.sanityn22 f117zc.sanityn23 f117zc.sanityn24 f117zc.sanityn25 f117zc.sanityn26 f117zc.sanityn27 f117zc.sanityn28 f117zc.sanityn29 f117zc.sanityn30 f117zc.sanityn31 f117zc.sanityn32 f117zc.sanityn33 f117zc.sanityn34 f117zc.sanityn35 f117zc.sanityn36 f117zc.sanityn37 f117zc.sanityn38 f117zc.sanityn39 f117zc.sanityn40 f117zc.sanityn41 f117zc.sanityn42 f117zc.sanityn43 f117zc.sanityn44 f117zc.sanityn45 f117zc.sanityn46 f117zc.sanityn47 f117zc.sanityn48 f117zc.sanityn49 f117zc.sanityn50 f117zc.sanityn51 f117zc.sanityn52 f117zc.sanityn53 f117zc.sanityn54 f117zc.sanityn55 f117zc.sanityn56 f117zc.sanityn57 f117zc.sanityn58 f117zc.sanityn59 f117zc.sanityn60 f117zc.sanityn61 f117zc.sanityn62 f117zc.sanityn63 f117zc.sanityn64 f117zc.sanityn65 f117zc.sanityn66 f117zc.sanityn67 f117zc.sanityn68 f117zc.sanityn69 f117zc.sanityn70 f117zc.sanityn71 f117zc.sanityn72 f117zc.sanityn73 f117zc.sanityn74 f117zc.sanityn75 f117zc.sanityn76 f117zc.sanityn77 f117zc.sanityn78 f117zc.sanityn79 f117zc.sanityn80 f117zc.sanityn81 f117zc.sanityn82 f117zc.sanityn83 f117zc.sanityn84 f117zc.sanityn85 f117zc.sanityn86 f117zc.sanityn87 f117zc.sanityn88 f117zc.sanityn89 f117zc.sanityn90 f117zc.sanityn91 f117zc.sanityn92 f117zc.sanityn93 f117zc.sanityn94 f117zc.sanityn95 f117zc.sanityn96 f117zc.sanityn97 f117zc.sanityn98 f117zc.sanityn99 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117zc (18s) == sanityn test 117zd: TCU: Test striped dir for UID Trash Can type with nodemap ========================================================== 04:42:12 (1766396532) On MGS 192.168.202.118, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.202.118, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid Creating 100 files under striped directory: /mnt/lustre/d117zd.sanityn/striped_dir total: 100 open/close in 0.92 seconds: 108.17 ops/second f117zd.sanityn0 -> 0 f117zd.sanityn1 -> 0 f117zd.sanityn2 -> 0 f117zd.sanityn3 -> 0 f117zd.sanityn4 -> 0 f117zd.sanityn5 -> 0 f117zd.sanityn6 -> 0 f117zd.sanityn7 -> 0 f117zd.sanityn8 -> 0 f117zd.sanityn9 -> 0 f117zd.sanityn10 -> 0 f117zd.sanityn11 -> 1 f117zd.sanityn12 -> 1 f117zd.sanityn13 -> 0 f117zd.sanityn14 -> 1 f117zd.sanityn15 -> 1 f117zd.sanityn16 -> 1 f117zd.sanityn17 -> 1 f117zd.sanityn18 -> 1 f117zd.sanityn19 -> 0 f117zd.sanityn20 -> 1 f117zd.sanityn21 -> 0 f117zd.sanityn22 -> 0 f117zd.sanityn23 -> 1 f117zd.sanityn24 -> 1 f117zd.sanityn25 -> 0 f117zd.sanityn26 -> 1 f117zd.sanityn27 -> 1 f117zd.sanityn28 -> 1 f117zd.sanityn29 -> 0 f117zd.sanityn30 -> 1 f117zd.sanityn31 -> 0 f117zd.sanityn32 -> 1 f117zd.sanityn33 -> 0 f117zd.sanityn34 -> 1 f117zd.sanityn35 -> 0 f117zd.sanityn36 -> 0 f117zd.sanityn37 -> 0 f117zd.sanityn38 -> 0 f117zd.sanityn39 -> 1 f117zd.sanityn40 -> 1 f117zd.sanityn41 -> 0 f117zd.sanityn42 -> 1 f117zd.sanityn43 -> 1 f117zd.sanityn44 -> 0 f117zd.sanityn45 -> 0 f117zd.sanityn46 -> 1 f117zd.sanityn47 -> 0 f117zd.sanityn48 -> 1 f117zd.sanityn49 -> 0 f117zd.sanityn50 -> 1 f117zd.sanityn51 -> 1 f117zd.sanityn52 -> 1 f117zd.sanityn53 -> 0 f117zd.sanityn54 -> 0 f117zd.sanityn55 -> 0 f117zd.sanityn56 -> 0 f117zd.sanityn57 -> 1 f117zd.sanityn58 -> 1 f117zd.sanityn59 -> 1 f117zd.sanityn60 -> 1 f117zd.sanityn61 -> 0 f117zd.sanityn62 -> 0 f117zd.sanityn63 -> 0 f117zd.sanityn64 -> 0 f117zd.sanityn65 -> 1 f117zd.sanityn66 -> 0 f117zd.sanityn67 -> 0 f117zd.sanityn68 -> 1 f117zd.sanityn69 -> 1 f117zd.sanityn70 -> 1 f117zd.sanityn71 -> 1 f117zd.sanityn72 -> 0 f117zd.sanityn73 -> 0 f117zd.sanityn74 -> 0 f117zd.sanityn75 -> 0 f117zd.sanityn76 -> 1 f117zd.sanityn77 -> 1 f117zd.sanityn78 -> 1 f117zd.sanityn79 -> 0 f117zd.sanityn80 -> 0 f117zd.sanityn81 -> 0 f117zd.sanityn82 -> 1 f117zd.sanityn83 -> 1 f117zd.sanityn84 -> 1 f117zd.sanityn85 -> 0 f117zd.sanityn86 -> 0 f117zd.sanityn87 -> 1 f117zd.sanityn88 -> 0 f117zd.sanityn89 -> 0 f117zd.sanityn90 -> 0 f117zd.sanityn91 -> 0 f117zd.sanityn92 -> 1 f117zd.sanityn93 -> 1 f117zd.sanityn94 -> 0 f117zd.sanityn95 -> 1 f117zd.sanityn96 -> 0 f117zd.sanityn97 -> 1 f117zd.sanityn98 -> 1 f117zd.sanityn99 -> 1 #0 -> 0x200001b70:0xe:0x0 #1 -> 0x2400013a0:0x4:0x0 - unlinked 0 (time 1766396565 ; total 0 ; last 0) total: 100 unlinks in 1 seconds: 100.000000 unlinks/second f117zd.sanityn0 f117zd.sanityn1 f117zd.sanityn10 f117zd.sanityn11 f117zd.sanityn12 f117zd.sanityn13 f117zd.sanityn14 f117zd.sanityn15 f117zd.sanityn16 f117zd.sanityn17 f117zd.sanityn18 f117zd.sanityn19 f117zd.sanityn2 f117zd.sanityn20 f117zd.sanityn21 f117zd.sanityn22 f117zd.sanityn23 f117zd.sanityn24 f117zd.sanityn25 f117zd.sanityn26 f117zd.sanityn27 f117zd.sanityn28 f117zd.sanityn29 f117zd.sanityn3 f117zd.sanityn30 f117zd.sanityn31 f117zd.sanityn32 f117zd.sanityn33 f117zd.sanityn34 f117zd.sanityn35 f117zd.sanityn36 f117zd.sanityn37 f117zd.sanityn38 f117zd.sanityn39 f117zd.sanityn4 f117zd.sanityn40 f117zd.sanityn41 f117zd.sanityn42 f117zd.sanityn43 f117zd.sanityn44 f117zd.sanityn45 f117zd.sanityn46 f117zd.sanityn47 f117zd.sanityn48 f117zd.sanityn49 f117zd.sanityn5 f117zd.sanityn50 f117zd.sanityn51 f117zd.sanityn52 f117zd.sanityn53 f117zd.sanityn54 f117zd.sanityn55 f117zd.sanityn56 f117zd.sanityn57 f117zd.sanityn58 f117zd.sanityn59 f117zd.sanityn6 f117zd.sanityn60 f117zd.sanityn61 f117zd.sanityn62 f117zd.sanityn63 f117zd.sanityn64 f117zd.sanityn65 f117zd.sanityn66 f117zd.sanityn67 f117zd.sanityn68 f117zd.sanityn69 f117zd.sanityn7 f117zd.sanityn70 f117zd.sanityn71 f117zd.sanityn72 f117zd.sanityn73 f117zd.sanityn74 f117zd.sanityn75 f117zd.sanityn76 f117zd.sanityn77 f117zd.sanityn78 f117zd.sanityn79 f117zd.sanityn8 f117zd.sanityn80 f117zd.sanityn81 f117zd.sanityn82 f117zd.sanityn83 f117zd.sanityn84 f117zd.sanityn85 f117zd.sanityn86 f117zd.sanityn87 f117zd.sanityn88 f117zd.sanityn89 f117zd.sanityn9 f117zd.sanityn90 f117zd.sanityn91 f117zd.sanityn92 f117zd.sanityn93 f117zd.sanityn94 f117zd.sanityn95 f117zd.sanityn96 f117zd.sanityn97 f117zd.sanityn98 f117zd.sanityn99 f117zd.sanityn10 f117zd.sanityn11 f117zd.sanityn12 f117zd.sanityn13 f117zd.sanityn14 f117zd.sanityn15 f117zd.sanityn16 f117zd.sanityn17 f117zd.sanityn18 f117zd.sanityn19 f117zd.sanityn20 f117zd.sanityn21 f117zd.sanityn22 f117zd.sanityn23 f117zd.sanityn24 f117zd.sanityn25 f117zd.sanityn26 f117zd.sanityn27 f117zd.sanityn28 f117zd.sanityn29 f117zd.sanityn30 f117zd.sanityn31 f117zd.sanityn32 f117zd.sanityn33 f117zd.sanityn34 f117zd.sanityn35 f117zd.sanityn36 f117zd.sanityn37 f117zd.sanityn38 f117zd.sanityn39 f117zd.sanityn40 f117zd.sanityn41 f117zd.sanityn42 f117zd.sanityn43 f117zd.sanityn44 f117zd.sanityn45 f117zd.sanityn46 f117zd.sanityn47 f117zd.sanityn48 f117zd.sanityn49 f117zd.sanityn50 f117zd.sanityn51 f117zd.sanityn52 f117zd.sanityn53 f117zd.sanityn54 f117zd.sanityn55 f117zd.sanityn56 f117zd.sanityn57 f117zd.sanityn58 f117zd.sanityn59 f117zd.sanityn60 f117zd.sanityn61 f117zd.sanityn62 f117zd.sanityn63 f117zd.sanityn64 f117zd.sanityn65 f117zd.sanityn66 f117zd.sanityn67 f117zd.sanityn68 f117zd.sanityn69 f117zd.sanityn70 f117zd.sanityn71 f117zd.sanityn72 f117zd.sanityn73 f117zd.sanityn74 f117zd.sanityn75 f117zd.sanityn76 f117zd.sanityn77 f117zd.sanityn78 f117zd.sanityn79 f117zd.sanityn80 f117zd.sanityn81 f117zd.sanityn82 f117zd.sanityn83 f117zd.sanityn84 f117zd.sanityn85 f117zd.sanityn86 f117zd.sanityn87 f117zd.sanityn88 f117zd.sanityn89 f117zd.sanityn90 f117zd.sanityn91 f117zd.sanityn92 f117zd.sanityn93 f117zd.sanityn94 f117zd.sanityn95 f117zd.sanityn96 f117zd.sanityn97 f117zd.sanityn98 f117zd.sanityn99 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid On MGS 192.168.202.118, active = nodemap.active=0 waiting 10 secs for sync PASS 117zd (53s) == sanityn test 118Aa: TCU: Orphan object handling for a regular file or an empty dir ========================================================== 04:43:05 (1766396585) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) 64003 (12) . 2 (4084) .. 520232 (4096) 0x200001b71:0x78:0x0 Waiting for MDT destroys to complete oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) 64003 (12) . 2 (4084) .. 0 (4096) 0x200001b71:0x78:0x0 sanityn test_118Aa: @@@@@@ FAIL: 0x200001b71:0x78:0x0 should be removed from PENDING Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9184:test_118Aa() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9206:main() Dumping lctl log to /tmp/testlogs//sanityn.test_118Aa.*.1766396607.log rsync: chown "/tmp/testlogs/.sanityn.test_118Aa.debug_log.oleg218-server.1766396607.log.fhty9q" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_118Aa.dmesg.oleg218-server.1766396607.log.WNtOiT" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid FAIL 118Aa (27s) == sanityn test 118Ab: TCU: Orphan object handling for non-emtpy dir ========================================================== 04:43:32 (1766396612) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) 520234 (4096) 0x200001b71:0x7a:0x0 Waiting for MDT destroys to complete oleg218-server: debugfs 1.47.3-wc2 (11-Nov-2025) 0 (4096) 0x200001b71:0x7a:0x0 sanityn test_118Ab: @@@@@@ FAIL: 0x200001b71:0x7a:0x0 should be removed/cleanup from PENDING Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9250:test_118Ab() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9254:main() Dumping lctl log to /tmp/testlogs//sanityn.test_118Ab.*.1766396627.log rsync: chown "/tmp/testlogs/.sanityn.test_118Ab.debug_log.oleg218-server.1766396627.log.CkxPTV" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_118Ab.dmesg.oleg218-server.1766396627.log.f4oCOr" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid FAIL 118Ab (18s) == sanityn test 118Ba: TCU: 'oldest' repeated deletion for a regular file or empty dir ========================================================== 04:43:51 (1766396631) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=oldest mdd.lustre-MDT0001.trash_can_delete_policy=oldest llite.lustre-ffff92ff2fe2d800.trash_file_read=1 llite.lustre-ffff92ff3d48e000.trash_file_read=1 llite.lustre-ffff92ff2fe2d800.trash_file_read=0 llite.lustre-ffff92ff3d48e000.trash_file_read=0 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118Ba (9s) == sanityn test 118Bb: TCU: 'timestamp' repeated deletion policy ========================================================== 04:44:00 (1766396640) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp f118Bb.sanityn f118Bb.sanityn.2025-12-22T09:44:05 f118Bb.sanityn.2025-12-22T09:44:05.273779150 f118Bb.sanityn.2025-12-22T09:44:05.309930975 f118Bb.sanityn.2025-12-22T09:44:05.340367681 f118Bb.sanityn.2025-12-22T09:44:05.369973768 f118Bb.sanityn.2025-12-22T09:44:05.401053941 f118Bb.sanityn.2025-12-22T09:44:05.433753652 f118Bb.sanityn.2025-12-22T09:44:05.460128962 f118Bb.sanityn.2025-12-22T09:44:05.486498988 f118Bb.sanityn.2025-12-22T09:44:05.518211645 f118Bb.sanityn.2025-12-22T09:44:05.554376649 f118Bb.sanityn.2025-12-22T09:44:05.589840871 f118Bb.sanityn.2025-12-22T09:44:05.631870072 f118Bb.sanityn.2025-12-22T09:44:05.675508709 f118Bb.sanityn.2025-12-22T09:44:05.719477161 d118Bb.sanityn d118Bb.sanityn.2025-12-22T09:44:06 d118Bb.sanityn.2025-12-22T09:44:06.659623442 d118Bb.sanityn.2025-12-22T09:44:06.687104603 d118Bb.sanityn.2025-12-22T09:44:06.708175458 d118Bb.sanityn.2025-12-22T09:44:06.726310473 d118Bb.sanityn.2025-12-22T09:44:06.744836028 d118Bb.sanityn.2025-12-22T09:44:06.763235885 d118Bb.sanityn.2025-12-22T09:44:06.783775391 d118Bb.sanityn.2025-12-22T09:44:06.811207853 d118Bb.sanityn.2025-12-22T09:44:06.834691330 d118Bb.sanityn.2025-12-22T09:44:06.860808589 d118Bb.sanityn.2025-12-22T09:44:06.886167092 d118Bb.sanityn.2025-12-22T09:44:06.905715010 d118Bb.sanityn.2025-12-22T09:44:06.925527991 d118Bb.sanityn.2025-12-22T09:44:06.947567695 d118Bb.sanityn.mulevel d118Bb.sanityn.mulevel.2025-12-22T09:44:08 d118Bb.sanityn.mulevel.2025-12-22T09:44:08.637814931 d118Bb.sanityn.mulevel.2025-12-22T09:44:08.854686832 d118Bb.sanityn.mulevel.2025-12-22T09:44:09 d118Bb.sanityn.mulevel.2025-12-22T09:44:09.298868069 d118Bb.sanityn.mulevel.2025-12-22T09:44:09.525272326 d118Bb.sanityn.mulevel.2025-12-22T09:44:09.755224497 d118Bb.sanityn.mulevel.2025-12-22T09:44:09.992606850 d118Bb.sanityn.mulevel.2025-12-22T09:44:10 d118Bb.sanityn.mulevel.2025-12-22T09:44:10.425236156 d118Bb.sanityn.mulevel.2025-12-22T09:44:10.666774404 d118Bb.sanityn.mulevel.2025-12-22T09:44:10.922194790 d118Bb.sanityn.mulevel.2025-12-22T09:44:11 d118Bb.sanityn.mulevel.2025-12-22T09:44:11.345937902 d118Bb.sanityn.mulevel.2025-12-22T09:44:11.594299353 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118Bb (18s) == sanityn test 118C: TCU: add support for rename() ====== 04:44:18 (1766396658) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp f118C.sanityn f118C.sanityn f118C.sanityn f118C.sanityn.2025-12-22T09:44:22 f118C.sanityn.2025-12-22T09:44:23 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118C (9s) == sanityn test 118d: TCU: repeated deletion of same fname for a empty dir in DNE env ========================================================== 04:44:27 (1766396667) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp d118d.sanityn.0 d118d.sanityn.0.2025-12-22T09:44:31 d118d.sanityn.0.2025-12-22T09:44:31.742255782 d118d.sanityn.0.2025-12-22T09:44:31.780816763 d118d.sanityn.0.2025-12-22T09:44:31.827881230 d118d.sanityn.0.2025-12-22T09:44:31.873873743 d118d.sanityn.0.2025-12-22T09:44:31.924067479 d118d.sanityn.0.2025-12-22T09:44:31.961542000 d118d.sanityn.0.2025-12-22T09:44:32 d118d.sanityn.0.2025-12-22T09:44:32.034081546 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118d (10s) == sanityn test 118e: TCU: Repeated deletion of same name for a non-empty remote dir ========================================================== 04:44:37 (1766396677) SKIP: sanityn test_118e needs 3 MDTs at least SKIP 118e (2s) == sanityn test 200: service remains healthy while able to process request ========================================================== 04:44:39 (1766396679) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=3 striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d200.sanityn fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1038 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 51.9065 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 84.6512 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 116.41 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 149.14 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 180.959 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg218-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92ff2fe2d800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff92ff2fe2d800.ost_server_uuid in IDLE state after 0 sec PASS 200 (198s) cleanup: ====================================================== == sanityn test complete, duration 8586 sec ============== 04:48:00 (1766396880) sanityn: FAIL: test_117yb /etc and /mnt/lustre/d117yb.sanityn/etc diff sanityn: FAIL: test_118Aa 0x200001b71:0x78:0x0 should be removed from PENDING sanityn: FAIL: test_118Ab 0x200001b71:0x7a:0x0 should be removed/cleanup from PENDING === sanityn: start cleanup 04:48:03 (1766396883) === Stopping clients: oleg218-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg218-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 04:52:15 (1766397135) ===