-----============= acceptance-small: sanityn ============----- Mon Dec 22 02:32:50 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg124-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a skipping tests SLOW=no: 33a === sanityn: start setup 02:33:03 (1766388783) === Starting client oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Mount client oleg124-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Started clients oleg124-client.virtnet: 192.168.201.124@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg124-client.virtnet: executing check_config_client /mnt/lustre oleg124-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg124-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9f6fd06fa000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9f6fe073e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6fd06fa000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6fe073e000.idle_timeout=debug disable quota as required oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanityn: finish setup 02:33:24 (1766388804) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143014 s, 73.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8201] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 02:33:25 (1766388805) PASS 0 (5s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 02:33:30 (1766388810) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (5s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 02:33:35 (1766388815) -rw-r--r-- 1 root root 0 Dec 22 02:33 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 02:33:39 (1766388819) -rw-r--r-- 1 root root 0 Dec 22 02:33 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 02:33:42 (1766388822) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 02:33:46 (1766388826) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 02:33:50 (1766388830) total 0 -rwxrwxrwx 1 root root 0 Dec 22 02:33 f2a -rwxrwxrwx 1 root root 0 Dec 22 02:33 f2b total 0 -rwxrwxrwx 1 root root 0 Dec 22 02:33 f2a -rwxrwxrwx 1 root root 0 Dec 22 02:33 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00793911 s, 64.5 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 02:33:54 (1766388834) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (4s) == sanityn test 2g: check blocks update on sync write ==== 02:33:59 (1766388839) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.137071 s, 15.3 MB/s /mnt/lustre/f2g.sanityn has 4096 blocks /mnt/lustre2/f2g.sanityn has 4096 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 02:34:02 (1766388842) PASS 3 (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 02:34:06 (1766388846) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 02:34:10 (1766388850) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (4s) == sanityn test 6: remove of open file on other node ============================================================================ 02:34:14 (1766388854) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 02:34:18 (1766388858) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 02:34:21 (1766388861) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 02:34:25 (1766388865) PASS 9a (4s) == sanityn test 9b: append to striped sparse file ======== 02:34:29 (1766388869) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336016 s, 31.2 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0353983 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (4s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 02:34:33 (1766388873) 1+0 records in 1+0 records out 1 byte copied, 0.00346222 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0126964 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00837148 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00723325 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0109268 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.013537 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0129578 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00666861 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0149976 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0132964 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00915108 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00902744 s, 0.1 kB/s PASS 10a (4s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 02:34:38 (1766388878) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0242328 s, 127 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0346166 s, 118 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.002741 s, 1.1 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 02:34:42 (1766388882) striped dir -i1 -c2 -H all_char /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 02:34:47 (1766388887) warning: '-runtime' deprecated, use '-t runtime' instead - link 548 (time 1766388898.44 total 10.01 last 54.76) - link 1053 (time 1766388908.44 total 20.01 last 50.48) - link 1591 (time 1766388918.45 total 30.02 last 53.77) - link 2215 (time 1766388928.46 total 40.03 last 62.32) - link 2811 (time 1766388938.46 total 50.03 last 59.59) - link 3535 (time 1766388948.47 total 60.03 last 72.38) - link 4233 (time 1766388958.48 total 70.05 last 69.70) - link 4932 (time 1766388968.48 total 80.05 last 69.89) - link 5679 (time 1766388978.49 total 90.06 last 74.65) total: 6334 link in 100.01 seconds: 63.33 ops/second using seed 431888648 running for 100 seconds - stat 10000 (time 1766388895 ; total 6 ; last 6) - stat 20000 (time 1766388902 ; total 13 ; last 7) - stat 30000 (time 1766388909 ; total 20 ; last 7) - stat 40000 (time 1766388915 ; total 26 ; last 6) - stat 50000 (time 1766388921 ; total 32 ; last 6) - stat 60000 (time 1766388927 ; total 38 ; last 6) - stat 70000 (time 1766388933 ; total 44 ; last 6) - stat 80000 (time 1766388940 ; total 51 ; last 7) - stat 87006 (time 1766388951 ; total 62 ; last 11) - stat 90000 (time 1766388955 ; total 66 ; last 4) - stat 96453 (time 1766388966 ; total 77 ; last 11) - stat 100000 (time 1766388971 ; total 82 ; last 5) - stat 106637 (time 1766388982 ; total 93 ; last 11) - stat 110000 (time 1766388986 ; total 97 ; last 4) total: 111687 stats in 100 seconds: 1116.869995 stats/second - unlinked 0 (time 1766388939 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile6334) error: No such file or directory total: 6333 unlinks in 91 seconds: 69.593407 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22665) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22667) - No such process PASS 12 (147s) == sanityn test 13: test directory page revocation ======= 02:37:13 (1766389033) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d13 f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 02:37:17 (1766389037) striped dir -i0 -c2 -H all_char /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 23875 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 02:37:21 (1766389041) striped dir -i0 -c2 -H all_char /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 24465 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 02:37:24 (1766389044) striped dir -i0 -c2 -H crush /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 25055 Terminated $DIR1/$tdir/sleep 60 PASS 14b (4s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 02:37:28 (1766389048) striped dir -i0 -c2 -H crush /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 25649 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 02:37:32 (1766389052) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 26244 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 02:37:36 (1766389056) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky Waiting for MDT destroys to complete 0 STRIPECOUNT=2 ORIGFREE=7207404 MAXFREE=2097152000 0 lustre-OST0000 avl=356 grnt=403 diff=-47 limit=46331 FULL lustre-OST0001 avl=2544 grnt=2891 diff=-347 limit=46331 FULL ERROR: dd blocks 7197038 != file size 7197060 LOG LOG2 file /tmp/oos2.log:dd: error writing '/mnt/lustre/oosfile': No space left on device /tmp/oos2.log:3594218+0 records in /tmp/oos2.log:3594217+0 records out /tmp/oos2.log:3680478208 bytes (3.7 GB, 3.4 GiB) copied, 484.769 s, 7.6 MB/s /tmp/oos2.log2:dd: error writing '/mnt/lustre2/oosfile2': No space left on device /tmp/oos2.log2:3602822+0 records in /tmp/oos2.log2:3602821+0 records out /tmp/oos2.log2:3689288704 bytes (3.7 GB, 3.4 GiB) copied, 484.903 s, 7.6 MB/s Waiting for MDT destroys to complete Waiting for MDT destroys to complete PASS 15 (526s) == sanityn test 16a: 2500 iterations of dual-mount fsx === 02:46:22 (1766389582) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0932497 s, 90.0 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0870076 s, 96.4 MB/s Chance of close/open is 1 in 50 Seed set to 9584 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn skipping zero length zero range skipping zero size read truncating to largest ever: 0x1d7fb3 fallocating to largest ever: 0x61e245 truncating to largest ever: 0x9613f1 000100[0] 1766389586.215311 mapread 0x1e28f8 thru 0x1e665a (0x3d63 bytes) fallocating to largest ever: 0x9e14da 000200[1] 1766389588.061391 mapread 0x0abeb3 thru 0x0b7028 (0xb176 bytes) 000300[1] 1766389589.952743 trunc from 0x49b4f5 to 0x49e55c (0x3068 bytes) 000400[0] 1766389591.863862 fallocate 0x660931 thru 0x665420 (0x4af0 bytes) fallocating to largest ever: 0xa00000 000500[1] 1766389593.828229 write 0x49575d thru 0x4a35fe (0xdea2 bytes) 000600[0] 1766389595.922205 read 0x034212 thru 0x0415ec (0xd3db bytes) 000800[0] 1766389599.508984 trunc from 0x0e7d21 to 0x56dc51 (0x485f31 bytes) 000900[0] 1766389601.492835 write 0x8f6445 thru 0x8fed81 (0x893d bytes) 001000[1] 1766389603.272278 fallocate 0x4daa7a thru 0x4dfc83 (0x520a bytes) 001100[0] 1766389605.120743 mapwrite 0x1818fb thru 0x188434 (0x6b3a bytes) 001400[0] 1766389609.758830 read 0x1aa596 thru 0x1aff79 (0x59e4 bytes) 001500[1] 1766389611.558934 write 0x779d00 thru 0x77c1eb (0x24ec bytes) 001600[0] 1766389613.415911 punch from 0x64d235 to 0x65987a (0xc646 bytes) 001700[0] 1766389615.403185 fallocate 0x12be3a thru 0x136783 (0xa94a bytes) 001900[1] 1766389619.404150 mapwrite 0x4395d8 thru 0x444567 (0xaf90 bytes) 002100[0] 1766389623.371202 mapread 0x0159fc thru 0x019baa (0x41af bytes) 002200[0] 1766389625.514778 trunc from 0x763b58 to 0x4f859b (0x26b5bc bytes) 002300[0] 1766389627.813000 read 0x0c60d6 thru 0x0c99ba (0x38e5 bytes) 002400[0] 1766389630.031692 trunc from 0x9c7b7b to 0x8540d4 (0x173aa6 bytes) 002500[0] 1766389632.051459 trunc from 0x9a9667 to 0x26f2ea (0x73a37c bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9632 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fallocating to largest ever: 0x110e53 truncating to largest ever: 0x82cb57 truncating to largest ever: 0x9a5c7e truncating to largest ever: 0x9fcbda 000100[0] 1766389634.142103 read 0x0f4000 thru 0x0ff52f (0xb530 bytes) 000200[0] 1766389635.617905 punch from 0x47875c to 0x47d35a (0x4bff bytes) fallocating to largest ever: 0xa00000 000400[0] 1766389638.627166 fallocate 0x838c6b thru 0x83ad11 (0x20a7 bytes) 000600[0] 1766389641.638904 fallocate 0x736ffc thru 0x7421b1 (0xb1b6 bytes) 000700[1] 1766389643.177022 fallocate 0x8a8fd6 thru 0x8afefe (0x6f29 bytes) 000800[0] 1766389644.768712 write_OD 0x1a8000 thru 0x1aafff (0x3000 bytes) 000900[1] 1766389646.211444 fallocate 0x1b7904 thru 0x1b8d3d (0x143a bytes) 001000[0] 1766389647.587244 punch from 0x56493e to 0x56b879 (0x6f3c bytes) 001100[1] 1766389649.413569 read 0x679000 thru 0x6841a4 (0xb1a5 bytes) 001200[0] 1766389650.987209 mapread 0x11e000 thru 0x1214db (0x34dc bytes) 001300[0] 1766389652.603621 trunc from 0x288df9 to 0x291c77 (0x8e7f bytes) 001400[0] 1766389654.436913 fallocate 0x188f49 thru 0x1913bd (0x8475 bytes) 001600[1] 1766389658.022990 mapread 0x154000 thru 0x15577f (0x1780 bytes) 001700[0] 1766389659.639324 mapread 0x480000 thru 0x48de79 (0xde7a bytes) 002100[1] 1766389666.016613 read 0x234000 thru 0x236c19 (0x2c1a bytes) 002300[1] 1766389669.388969 punch from 0x3b1ad1 to 0x3b1e3d (0x36d bytes) 002400[0] 1766389670.760665 trunc from 0x0031e7 to 0x01122e (0xe048 bytes) All operations completed A-OK! PASS 16a (92s) == sanityn test 16b: 2500 iterations of dual-mount fsx at small size ========================================================== 02:47:54 (1766389674) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.107201 s, 78.3 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0970729 s, 86.4 MB/s Chance of close/open is 1 in 50 Seed set to 9675 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fallocating to largest ever: 0x9912d skipping zero size read skipping zero size read truncating to largest ever: 0x564e08 fallocating to largest ever: 0x67fc25 fallocating to largest ever: 0x8f61ac truncating to largest ever: 0x95751d fallocating to largest ever: 0x9fc584 000100[1] 1766389677.904700 mapwrite 0x3b8867 thru 0x3b8e87 (0x621 bytes) 000300[1] 1766389681.439997 punch from 0x5b7eb0 to 0x5b80c2 (0x213 bytes) 000400[1] 1766389683.010728 mapread 0x47a3d4 thru 0x47a8b8 (0x4e5 bytes) 000600[1] 1766389686.429643 mapread 0x21c745 thru 0x21d305 (0xbc1 bytes) 000700[1] 1766389688.126806 mapwrite 0x3ce534 thru 0x3cf2ba (0xd87 bytes) 000800[1] 1766389690.313897 trunc from 0x10f22b to 0x10fa05 (0x7db bytes) 000900[0] 1766389691.921909 read 0x08e796 thru 0x090611 (0x1e7c bytes) 001000[0] 1766389693.426005 trunc from 0x1d615b to 0x490af0 (0x2ba996 bytes) 001100[1] 1766389694.894166 read 0x379627 thru 0x379baa (0x584 bytes) 001200[1] 1766389696.661997 mapwrite 0x0ac165 thru 0x0ad866 (0x1702 bytes) 001300[1] 1766389698.624063 mapread 0x17ac70 thru 0x17c51d (0x18ae bytes) 001400[1] 1766389700.763059 trunc from 0x002995 to 0x002c27 (0x293 bytes) 001600[1] 1766389708.088935 trunc from 0x0d6de4 to 0x0d716f (0x38c bytes) 001700[1] 1766389712.751948 mapread 0x44c6cf thru 0x44c754 (0x86 bytes) 001800[1] 1766389716.670301 punch from 0x035921 to 0x0373cf (0x1aaf bytes) 001900[1] 1766389721.912423 mapread 0x2b11b1 thru 0x2b17d7 (0x627 bytes) 002000[0] 1766389726.040886 mapwrite 0x638438 thru 0x6393e2 (0xfab bytes) 002100[0] 1766389729.632881 trunc from 0x941672 to 0x47cd16 (0x4c495b bytes) 002200[0] 1766389733.567810 read 0x468b42 thru 0x46a529 (0x19e8 bytes) 002300[0] 1766389740.802020 fallocate 0x85848d thru 0x858753 (0x2c7 bytes) 002500[0] 1766389752.388126 read 0x0adbe0 thru 0x0ae706 (0xb27 bytes) All operations completed A-OK! PASS 16b (86s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 02:49:20 (1766389760) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.367531 s, 22.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.224812 s, 37.3 MB/s Waiting for MDT destroys to complete Chance of close/open is 1 in 50 Seed set to 9789 fd 0: /mnt/lustre/f16c.sanityn fd 1: /mnt/lustre2/f16c.sanityn zero_range to largest ever: 0x41cfc3 truncating to largest ever: 0x9999a1 fallocating to largest ever: 0x9d88c1 000100[1] 1766389794.374299 write 0x9357bd thru 0x940d50 (0xb594 bytes) 000200[1] 1766389798.480120 mapwrite 0x84e6af thru 0x84edf0 (0x742 bytes) 000300[1] 1766389802.944334 mapread 0x28092e thru 0x28d2aa (0xc97d bytes) 000400[0] 1766389807.356436 read 0x832849 thru 0x8410d8 (0xe890 bytes) 000500[0] 1766389811.622645 mapread 0x07e4e8 thru 0x08195d (0x3476 bytes) 000700[0] 1766389821.628331 write 0x17ca5c thru 0x18977f (0xcd24 bytes) 000800[0] 1766389826.758551 trunc from 0x7e7417 to 0x7f376b (0xc355 bytes) 001000[0] 1766389835.578312 trunc from 0x7afc9e to 0x4c9cce (0x2e5fcf bytes) 001200[0] 1766389844.032346 trunc from 0x2dadb7 to 0x2dd6fc (0x2946 bytes) truncating to largest ever: 0x9f7353 001300[1] 1766389847.672284 read 0x5cc619 thru 0x5cf046 (0x2a2e bytes) 001400[0] 1766389851.777370 trunc from 0x962797 to 0x001983 (0x960e13 bytes) 001500[1] 1766389856.723112 trunc from 0x804b7d to 0x119566 (0x6eb616 bytes) truncating to largest ever: 0x9f97df 001700[1] 1766389865.253308 fallocate 0x5ed57d thru 0x5f12b9 (0x3d3d bytes) 001800[1] 1766389869.681925 mapread 0x691ca5 thru 0x696f37 (0x5293 bytes) 001900[1] 1766389874.456850 trunc from 0x27940a to 0x2821a6 (0x8d9d bytes) 002000[1] 1766389878.536377 trunc from 0x57079f to 0x77b31e (0x20ab80 bytes) 002100[1] 1766389883.466410 trunc from 0x5b8bcd to 0x0c6173 (0x4f2a59 bytes) 002200[1] 1766389887.381251 trunc from 0x5bacd1 to 0x6eae04 (0x130134 bytes) 002300[0] 1766389891.769744 trunc from 0x0d04ad to 0x0df3ff (0xef53 bytes) 002400[1] 1766389896.030885 punch from 0x738614 to 0x742010 (0x99fd bytes) 002500[0] 1766389899.836077 mapread 0x4bfaa6 thru 0x4cc6f9 (0xcc54 bytes) All operations completed A-OK! PASS 16c (144s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 02:51:44 (1766389904) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.129646 s, 64.7 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.156185 s, 53.7 MB/s ldlm.namespaces.MGC192.168.201.124@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9f6fd06fa000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9f6fe073e000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9f6fd06fa000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9f6fe073e000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9f6fe073e000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9f6fe073e000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.48669 s, 23.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.17534 s, 48.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.56443 s, 67.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.63386 s, 28.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.02347 s, 34.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.20162 s, 47.6 MB/s PASS 16d (25s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 02:52:09 (1766389929) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.138162 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287084 s, 36.5 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 02:52:12 (1766389932) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 02:52:36 (1766389956) TIMEOUT PASS 16g (25s) == sanityn test 16h: mmap read after truncate file ======= 02:53:01 (1766389981) striped dir -i0 -c2 -H all_char /mnt/lustre/d16h.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (6s) == sanityn test 16i: read after truncate file ============ 02:53:07 (1766389987) striped dir -i0 -c2 -H all_char /mnt/lustre/d16i.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (6s) == sanityn test 16j: race dio with buffered i/o ========== 02:53:13 (1766389993) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.662229 s, 39.6 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.792294 s, 33.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.13047 s, 23.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.703436 s, 37.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.06252 s, 24.7 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.802654 s, 32.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.25368 s, 20.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.10873 s, 23.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.27034 s, 20.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.31108 s, 20.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.45556 s, 18.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.21127 s, 21.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.23172 s, 21.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.52923 s, 17.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.60153 s, 16.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.87282 s, 30.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.04144 s, 25.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.52164 s, 17.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.50647 s, 17.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (22s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 02:53:35 (1766390015) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fallocating to largest ever: 0x3428364 fallocating to largest ever: 0x3428364 skipping zero size read skipping zero size read fallocating to largest ever: 0x3428364 fallocating to largest ever: 0x3428364 skipping zero size read skipping zero size read fallocating to largest ever: 0x3428364 skipping zero size read truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Mon Dec 22 02:54:01 EST 2025 + fsx 2 done: rc=0 Mon Dec 22 02:54:01 EST 2025 + fsx 3 done: rc=0 Mon Dec 22 02:54:01 EST 2025 All operations completed A-OK! + fsx 4 done: rc=0 Mon Dec 22 02:54:01 EST 2025 All operations completed A-OK! + fsx 5 done: rc=0 Mon Dec 22 02:54:01 EST 2025 PASS 16k (33s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 02:54:08 (1766390048) fail_loc=0x8000030a PASS 17 (8s) == sanityn test 18: mmap sanity check =========================================================================================== 02:54:16 (1766390056) mmap test1: basic mmap operation (PASS, 0.12193s) mmap test2: MAP_PRIVATE not write back (PASS, 0.075932s) mmap test3: concurrent mmap ops on two nodes (PASS, 31.425s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.5171s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.19046s) mmap test6: check mmap write/read content on two nodes (PASS, 0.22642s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.79684s) mmap test11: multi-thread shared mmap (PASS, 0.63361s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0655s) PASS 18 (45s) == sanityn test 19: test concurrent uncached read races ========================================================================= 02:55:02 (1766390102) 32+0 records in 32+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.328064 s, 51.1 MB/s loop 5 loop 10 loop 15 loop 20 PASS 19 (21s) == sanityn test 20: test extra readahead page left in cache ============================================================== 02:55:22 (1766390122) striped dir -i0 -c2 -H crush /mnt/lustre/d20.sanityn PASS 20 (5s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 02:55:27 (1766390127) striped dir -i1 -c2 -H crush /mnt/lustre/d21.sanityn rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 02:55:31 (1766390131) now is 1766390133 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 new atime is 1766390194 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 02:56:36 (1766390196) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 8732 3598288 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 30700 3576320 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 39432 7174608 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 8732 3598288 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 30700 3576320 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 39432 7174608 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 351 999.7K 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1000.0K 281 999.7K 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 256.0K 402 255.6K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 256.0K 403 255.6K 1% /mnt/lustre[OST:1] filesystem_summary: 511.8K 632 511.2K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 351 999.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1000.0K 281 999.7K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 256.0K 402 255.6K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 256.0K 403 255.6K 1% /mnt/lustre2[OST:1] filesystem_summary: 511.8K 632 511.2K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 1.3G 2.0M 1.2G 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1.3G 1.7M 1.2G 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3.7G 8.5M 3.4G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.7G 30.0M 3.4G 1% /mnt/lustre[OST:1] filesystem_summary: 7.3G 38.5M 6.8G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 351 1023649 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1024000 281 1023719 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 262144 402 261742 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 262144 403 261741 1% /mnt/lustre2[OST:1] filesystem_summary: 524115 632 523483 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 8732 3598288 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 30700 3576320 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 39432 7174608 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 351 999.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1000.0K 281 999.7K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 256.0K 402 255.6K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 256.0K 403 255.6K 1% /mnt/lustre2[OST:1] filesystem_summary: 511.8K 632 511.2K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 351 1023649 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1024000 281 1023719 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 262144 402 261742 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 262144 403 261741 1% /mnt/lustre[OST:1] filesystem_summary: 524115 632 523483 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 351 1023649 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1024000 281 1023719 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 262144 402 261742 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 262144 403 261741 1% /mnt/lustre2[OST:1] filesystem_summary: 524115 632 523483 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 8732 3598288 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 30700 3576320 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 39432 7174608 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2036 1269268 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 8732 3598288 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 30700 3576320 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 39432 7174608 1% /mnt/lustre2 PASS 24a (4s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 02:56:40 (1766390200) PASS 24b (4s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 02:56:44 (1766390204) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (4s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 02:56:48 (1766390208) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] PASS 25b (4s) == sanityn test 26a: allow mtime to get older ============ 02:56:53 (1766390213) utime: good mknod times 1766390213 <= 1766390214 <= 1766390214 for /mnt/lustre/f26a utime: good mknod times 1766390213 <= 1766390214 <= 1766390214 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (4s) == sanityn test 26b: sync mtime between ost and mds ====== 02:56:57 (1766390217) PASS 26b (6s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 02:57:03 (1766390223) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289279612 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-12-22 02:57:04.000000000 -0500 Birth: 2025-12-22 02:57:04.000000000 -0500 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289279612 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-12-22 02:57:04.000000000 -0500 Birth: 2025-12-22 02:57:04.000000000 -0500 PASS 26c (5s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 02:57:09 (1766390229) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d30.sanityn 0 PASS 30 (6s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 02:57:15 (1766390235) striped dir -i1 -c2 -H crush /mnt/lustre/d31a.sanityn fail_loc=0x314 PASS 31a (4s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 02:57:19 (1766390239) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg124-server mds-ost sync done. Waiting for MDT destroys to complete striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d31b.sanityn fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.124@tcp:/lustre 7666232 40560 7173480 1% /mnt/lustre2 PASS 31b (11s) == sanityn test 31r: open-rename(replace) race =========== 02:57:30 (1766390250) -rw-r--r-- 1 root root 0 Dec 22 02:57 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (7s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 02:57:37 (1766390257) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 47565 Terminated tail -f $tfile PASS 31s (4s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 02:57:41 (1766390261) PASS 31t (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 02:57:47 (1766390267) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 02:57:49 (1766390269) Stopping /mnt/lustre-mds1 (opts:) on oleg124-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0000 create file on 2nd stripe under striped directory /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 2 /mnt/lustre/d33c.sanityn/striped touch /mnt/lustre2/d33c.sanityn/striped/subfile SoLC count 2 Below operations shouldn't trigger Solc: create file on 2nd stripe after setattr chmod 777 /mnt/lustre/d33c.sanityn/striped; touch /mnt/lustre2/d33c.sanityn/striped/subfile create remote dir and local dir /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33c.sanityn/remote create local dir after remote dir creation transaction commit /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33c.sanityn/remote.3; do_facet mds2 /home/green/git/lustre-release/lustre/utils/lctl set_param -n osd*.*MDT0001.force_sync 1; mkdir /mnt/lustre/d33c.sanityn/local.3 Solc count 0 PASS 33c (28s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 02:58:17 (1766390297) remote directory create /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre2/d33d.sanityn/subdir CoS count 2 cross-MDT rename mkdir /mnt/lustre/d33d.sanityn; /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn/d1; /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33d.sanityn/d2; touch /mnt/lustre/d33d.sanityn/d1/src mv /mnt/lustre2/d33d.sanityn/d1/src /mnt/lustre2/d33d.sanityn/d2/tgt CoS count 2 migrate /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn /home/green/git/lustre-release/lustre/utils/lfs migrate -m 1 /mnt/lustre2/d33d.sanityn CoS count 2 PASS 33d (25s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 02:58:42 (1766390322) plain dir creation mkdir /mnt/lustre2/d33e.sanityn/plain open file and write echo abc > /mnt/lustre2/d33e.sanityn/f33e.sanityn append write echo dfg >> /mnt/lustre2/d33e.sanityn/f33e.sanityn setattr touch /mnt/lustre2/d33e.sanityn/f33e.sanityn file unlink rm /mnt/lustre2/d33e.sanityn/f33e.sanityn plain dir unlink rmdir /mnt/lustre2/d33e.sanityn/plain striped directory creation /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 2 /mnt/lustre2/d33e.sanityn/striped set default LMV to create striped subdir /home/green/git/lustre-release/lustre/utils/lfs setdirstripe -D -c 2 /mnt/lustre/d33e.sanityn striped subdir creation createmany -d /mnt/lustre/d33e.sanityn/subdir 100 total: 100 mkdir in 0.85 seconds: 117.24 ops/second sub file creation and write createmany -o /mnt/lustre/d33e.sanityn/subfile 100; echo abc > /mnt/lustre/d33e.sanityn/subfile1 total: 100 open/close in 0.96 seconds: 104.07 ops/second sub file append write echo dfg >> /mnt/lustre2/d33e.sanityn/subfile2 subdir setatttr touch /mnt/lustre2/d33e.sanityn/subdir1 subdir unlink unlinkmany -d /mnt/lustre/d33e.sanityn/subdir 100 - unlinked 0 (time 1766390325 ; total 0 ; last 0) total: 100 unlinks in 4 seconds: 25.000000 unlinks/second sub file unlink unlinkmany /mnt/lustre2/d33e.sanityn/subfile 100 - unlinked 0 (time 1766390329 ; total 0 ; last 0) total: 100 unlinks in 1 seconds: 100.000000 unlinks/second sub file creation follows striped dir chmod chmod 777 /mnt/lustre/d33e.sanityn/striped; touch /mnt/lustre/d33e.sanityn/striped/subfile striped directory unlink rm -rf /mnt/lustre2/d33e.sanityn/striped directory unlink rm -rf /mnt/lustre2/d33e.sanityn CoS count 0 PASS 33e (11s) == sanityn test 34: no lock timeout under IO ============= 02:58:53 (1766390333) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f6fd06fa000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9f6fd06fa000.ost_server_uuid in FULL state after 0 sec oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f6fd06fa000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9f6fd06fa000.ost_server_uuid in IDLE state after 0 sec oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f6fd06fa000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9f6fd06fa000.ost_server_uuid in IDLE state after 0 sec oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f6fd06fa000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9f6fd06fa000.ost_server_uuid in IDLE state after 0 sec oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f6fd06fa000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9f6fd06fa000.ost_server_uuid in IDLE state after 0 sec oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f6fd06fa000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9f6fd06fa000.ost_server_uuid in IDLE state after 0 sec PASS 34 (98s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 03:00:31 (1766390431) striped dir -i1 -c2 -H crush2 /mnt/lustre/d35.sanityn fail_loc=0x80000317 Race attempt 0 Wait for 57555 57645 for 60 sec... - open/close 1109 (time 1766390445.65 total 10.00 last 110.86) - open/close 2171 (time 1766390455.65 total 20.00 last 106.20) - open/close 3213 (time 1766390465.66 total 30.02 last 104.08) total: 4000 open/close in 36.35 seconds: 110.05 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.124@tcp:/lustre 7.4G 40M 6.9G 1% /mnt/lustre 192.168.201.124@tcp:/lustre 7.4G 40M 6.9G 1% /mnt/lustre2 PASS 35 (67s) == sanityn test 36: handle ESTALE/open-unlink correctly == 03:01:38 (1766390498) striped dir -i0 -c2 -H crush /mnt/lustre/d36.sanityn Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg124-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.590365 s, 88.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(0) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.61184 s, 85.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(1) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.694543 s, 75.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(2) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.564774 s, 92.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(3) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.606062 s, 86.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(4) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.577888 s, 90.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(5) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.674643 s, 77.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(6) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.554675 s, 94.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(7) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.713932 s, 73.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(8) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.577042 s, 90.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(9) *** before(7173424) after_dd(7122224) after(7173424) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.543794 s, 96.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Waiting for MDT destroys to complete *** cycle(10) *** before(7173424) after_dd(7122224) after(7173424) PASS 36 (181s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 03:04:39 (1766390679) striped dir -i1 -c2 -H crush2 /mnt/lustre/d37.sanityn multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 - create 5316 (time 1766390691.46 total 10.00 last 531.49) total: 10000 create in 16.72 seconds: 598.02 ops/second PASS 37 (21s) == sanityn test 39a: file mtime does not change after rename ========================================================== 03:05:00 (1766390700) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 03:05:03 (1766390703) repeat after cancel_lru_locks PASS 39b (4s) == sanityn test 39c: check truncate mtime update ================================================================================ 03:05:07 (1766390707) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 03:05:10 (1766390710) fail_loc=0x411 fail_loc=0 PASS 39d (2s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 03:05:13 (1766390713) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (9s) == sanityn test 40c: pdirops: link and others ======================================================================== 03:05:22 (1766390722) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=7 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 03:05:30 (1766390730) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (9s) == sanityn test 40e: pdirops: rename and others ======================================================================== 03:05:39 (1766390739) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 03:05:46 (1766390746) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (6s) == sanityn test 41b: pdirops: create vs create ======================================================================== 03:05:52 (1766390752) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 03:05:58 (1766390758) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 03:06:04 (1766390764) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 03:06:09 (1766390769) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 41e (6s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 03:06:15 (1766390775) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=7 Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 03:06:21 (1766390781) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 03:06:26 (1766390786) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 Conflict PASS 41h (6s) == sanityn test 41i: reint_open: create vs create ======== 03:06:32 (1766390792) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (747s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 03:18:59 (1766391539) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (15s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 03:19:14 (1766391554) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 42b (18s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 03:19:32 (1766391572) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (14s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 03:19:46 (1766391586) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42d (9s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 03:19:55 (1766391595) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (10s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 03:20:05 (1766391605) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42f (8s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 03:20:13 (1766391613) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42g (8s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 03:20:21 (1766391621) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 42h (10s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 03:20:31 (1766391631) PASS 43a (84s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 03:21:55 (1766391715) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43b (10s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 03:22:05 (1766391725) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43c (10s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 03:22:15 (1766391735) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (9s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 03:22:25 (1766391745) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43e (11s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 03:22:36 (1766391756) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (10s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 03:22:46 (1766391766) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (11s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 03:22:57 (1766391777) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43h (10s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 03:23:07 (1766391787) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43i (10s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 03:23:17 (1766391797) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (110s) == sanityn test 43k: unlink vs create ==================== 03:25:07 (1766391907) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (882s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 03:39:49 (1766392789) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon Dec 22 03:39:54 EST 2025 PASS 44a (8s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 03:39:57 (1766392797) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 open: File exists Conflict PASS 44b (7s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 03:40:04 (1766392804) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (8s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 03:40:12 (1766392812) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44d (8s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 03:40:20 (1766392820) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44e (8s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 03:40:28 (1766392828) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=8 Conflict PASS 44f (7s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 03:40:35 (1766392835) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44g (7s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 03:40:42 (1766392842) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44h (8s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 03:40:50 (1766392850) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 lfs mkdir: dirstripe error on '/mnt/lustre2/f44i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f44i.sanityn': File exists Conflict PASS 44i (7s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 03:40:57 (1766392857) PASS 45a (92s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 03:42:30 (1766392950) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45b (7s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 03:42:37 (1766392957) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 Conflict PASS 45c (8s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 03:42:45 (1766392965) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (7s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 03:42:53 (1766392973) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45e (9s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 03:43:01 (1766392981) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (8s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 03:43:09 (1766392989) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (7s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 03:43:17 (1766392997) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 45h (8s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 03:43:24 (1766393004) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45i (7s) == sanityn test 45j: read vs rename ====================== 03:43:31 (1766393011) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (594s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 03:53:25 (1766393605) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 03:53:30 (1766393610) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 open: File exists Conflict PASS 46b (7s) == sanityn test 46c: pdirops: link vs link ======================================================================== 03:53:37 (1766393617) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 03:53:42 (1766393622) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 46d (6s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 03:53:48 (1766393628) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 46e (6s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 03:53:54 (1766393634) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=8 Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 03:54:00 (1766393640) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=8 Conflict PASS 46g (6s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 03:54:06 (1766393646) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 46h (6s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 03:54:12 (1766393652) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 lfs mkdir: dirstripe error on '/mnt/lustre2/f46i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f46i.sanityn': File exists Conflict PASS 46i (6s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 03:54:18 (1766393658) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 mkdir: cannot create directory '/mnt/lustre2/f47a.sanityn': File exists Conflict PASS 47a (6s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 03:54:24 (1766393664) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 open: File exists Conflict PASS 47b (7s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 03:54:31 (1766393671) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f47c.sanityn' to '/mnt/lustre2/f47c.sanityn-2': File exists Conflict PASS 47c (6s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 03:54:37 (1766393677) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 47d (6s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 03:54:43 (1766393683) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f47e.sanityn' with non-directory Conflict PASS 47e (6s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 03:54:49 (1766393689) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 47f (7s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 03:54:56 (1766393696) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47g (7s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 03:55:03 (1766393703) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00556982 s, 1.8 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 03:55:11 (1766393711) 0+1 records in 0+1 records out 158 bytes copied, 0.00769352 s, 20.5 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3510: kill: (284832) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 03:55:15 (1766393715) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00304529 s, 336 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236324 s, 44.4 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 03:55:30 (1766393730) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00527106 s, 194 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 03:55:37 (1766393737) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180415 s, 58.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 03:55:40 (1766393740) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 1312 0x520 0x280000401 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 1312 0x520 0x280000401 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 03:55:45 (1766393745) llite.lustre-ffff9f6fd06fa000.xattr_cache=1 llite.lustre-ffff9f6fe073e000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762674 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:55:46.000000000 -0500 Modify: 2025-12-22 03:55:46.000000000 -0500 Change: 2025-12-22 03:55:46.000000000 -0500 Birth: 2025-12-22 03:55:46.000000000 -0500 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281279 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:55:51.000000000 -0500 Modify: 2025-12-22 03:55:52.000000000 -0500 Change: 2025-12-22 03:55:52.000000000 -0500 Birth: 2025-12-22 03:55:52.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762677 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:55:57.000000000 -0500 Modify: 2025-12-22 03:55:57.000000000 -0500 Change: 2025-12-22 03:55:57.000000000 -0500 Birth: 2025-12-22 03:55:57.000000000 -0500 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281280 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:55:57.000000000 -0500 Modify: 2025-12-22 03:55:57.000000000 -0500 Change: 2025-12-22 03:55:57.000000000 -0500 Birth: 2025-12-22 03:55:57.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281282 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:56:02.000000000 -0500 Modify: 2025-12-22 03:56:03.000000000 -0500 Change: 2025-12-22 03:56:03.000000000 -0500 Birth: 2025-12-22 03:56:03.000000000 -0500 File: /mnt/lustre/d1 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762678 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:56:02.000000000 -0500 Modify: 2025-12-22 03:56:03.000000000 -0500 Change: 2025-12-22 03:56:03.000000000 -0500 Birth: 2025-12-22 03:56:03.000000000 -0500 PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 03:56:10 (1766393770) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 03:56:18 (1766393778) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 03:56:26 (1766393786) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 03:56:39 (1766393799) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 03:56:49 (1766393809) fail_loc=0x155 We survived after AB/BA race test fail_loc=0x0 PASS 55e (14s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 03:57:03 (1766393823) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (13s) == sanityn test 55g: rename: race with trylock =========== 03:57:16 (1766393836) mdt.lustre-MDT0000.enable_rename_trylock=1 mdt.lustre-MDT0001.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre/d55g.sanityn/a1/a2' to a subdirectory of itself, '/mnt/lustre/d55g.sanityn/b1/b2/b3/b4/a2' We survived race with trylock test fail_loc=0x0 PASS 55g (15s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 03:57:31 (1766393851) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1766393852 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 12.2078 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (8s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 03:57:39 (1766393859) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1766393861 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.69516 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (31s) == sanityn test 60: Verify data_version behaviour ======== 03:58:10 (1766393890) striped dir -i0 -c2 -H all_char /mnt/lustre/d60.sanityn PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 03:58:13 (1766393893) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 03:58:16 (1766393896) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 03:58:18 (1766393898) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00357141 s, 18.4 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00297553 s, 22.0 MB/s File: /mnt/lustre2/f71a.sanityn Size: 262144 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289281322 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:58:19.000000000 -0500 Modify: 2025-12-22 03:58:19.000000000 -0500 Change: 2025-12-22 03:58:19.000000000 -0500 Birth: 2025-12-22 03:58:19.000000000 -0500 25929 extent 0 in offset 65536, length 65536 flags (0x80000000): NET extent 1 in offset 196608, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 2, sum of lengths 131072, expected sum 131072 25930 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00327471 s, 20.0 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0028279 s, 23.2 MB/s File: /mnt/lustre/f71a.sanityn Size: 262144 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289281322 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 03:58:19.000000000 -0500 Modify: 2025-12-22 03:58:19.000000000 -0500 Change: 2025-12-22 03:58:19.000000000 -0500 Birth: 2025-12-22 03:58:19.000000000 -0500 25931 extent 0 in offset 65536, length 65536 flags (0x80000000): NET extent 1 in offset 196608, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 2, sum of lengths 131072, expected sum 131072 25930 PASS 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 03:58:21 (1766393901) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00467664 s, 14.0 MB/s extent 0 in offset 0, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 1, sum of lengths 65536, expected sum 65536 PASS 71b (2s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 03:58:23 (1766393903) filefrag 1.47.3-wc2 (11-Nov-2025) check last flag for file with 291 extents 290: 296960.. 296963: 583680.. 583683: 4: 0000: last,net,eof check last flag for file with 292 extents 291: 297984.. 297987: 338944.. 338947: 4: 0001: last,net,eof check last flag for file with 293 extents 292: 299008.. 299011: 839680.. 839683: 4: 0000: last,net,eof PASS 71c (13s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 03:58:36 (1766393916) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00960751 s, 6.8 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00191856 s, 34.2 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00145242 s, 45.1 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0148054 s, 4.4 MB/s disk usage: 262144 /mnt/lustre/f71d.sanityn file size: 458752 /mnt/lustre/f71d.sanityn extent 0 in offset 0, length 65536 flags (0x80000000): NET extent 1 in offset 131072, length 65536 flags (0x80000000): NET extent 2 in offset 262144, length 65536 flags (0x80000000): NET extent 3 in offset 393216, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 4, sum of lengths 262144, expected sum 262144 PASS 71d (15s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 03:58:51 (1766393931) llite.lustre-ffff9f6fd06fa000.xattr_cache=1 llite.lustre-ffff9f6fe073e000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 03:58:54 (1766393934) llite.lustre-ffff9f6fd06fa000.xattr_cache=1 llite.lustre-ffff9f6fe073e000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (5s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 03:58:59 (1766393939) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0173391 s, 59.1 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00926294 s, 111 kB/s 300333: get lock2 300334: get lock1 300334: done 300334: get lock2 300333: done 300333: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 300333: cannot get lock1: Resource deadlock avoided 300334: done 300334: exit rc=0 300333: done 300333: exit rc=0 PASS 74 (9s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 03:59:08 (1766393948) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0345388 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251689 s, 41.7 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4193: 300924 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (10s) == sanityn test 76: Verify MDT open_files listing ======== 03:59:18 (1766393958) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d76.sanityn opening files: ................ closing files: ................ PASS 76 (70s) == sanityn test 77a: check FIFO NRS policy =============== 04:00:29 (1766394029) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.187182 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274713 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425116 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051207 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131552 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136227 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11501 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136806 s, 7.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11905 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133299 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119789 s, 8.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144454 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127489 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152837 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155319 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153175 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151165 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00306402 s, 342 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586394 s, 17.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518047 s, 20.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433744 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00240983 s, 435 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590275 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510445 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037489 s, 28.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329423 s, 31.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00259071 s, 405 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267766 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235278 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208668 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177939 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015487 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013566 s, 77.3 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (5s) == sanityn test 77b: check CRR-N NRS policy ============== 04:00:33 (1766394033) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.215726 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372837 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393036 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857595 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114266 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144059 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137642 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134541 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118638 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153498 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136328 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149158 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167181 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166695 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172003 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147975 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167782 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402139 s, 261 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00461768 s, 227 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667229 s, 157 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859009 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333166 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00336222 s, 312 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400071 s, 26.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049275 s, 21.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606131 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0039398 s, 266 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193381 s, 54.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280233 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170034 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149746 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140699 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013586 s, 77.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=5 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.195623 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344724 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327246 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120202 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125088 s, 8.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113794 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135944 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139442 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127705 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120976 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136221 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145921 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145219 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141676 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161373 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157407 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152547 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227365 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335561 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238287 s, 44.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365242 s, 28.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248937 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234166 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192563 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488532 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481375 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303817 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353916 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473728 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0492985 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458895 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311258 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359915 s, 29.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=8 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=3 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 04:00:38 (1766394038) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.220135 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381729 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477428 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042464 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997232 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122917 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105607 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12221 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127127 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116968 s, 9.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134048 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130164 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127294 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134119 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143804 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141032 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137666 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150773 s, 69.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209782 s, 50.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213219 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338167 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370311 s, 28.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274289 s, 38.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236604 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245782 s, 42.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407843 s, 25.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242356 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02673 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304426 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270941 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462457 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421505 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314394 s, 33.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.181242 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338324 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377055 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103605 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104514 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11631 s, 9.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121873 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133885 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132589 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146213 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128613 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140127 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144582 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13421 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133208 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129598 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128636 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294355 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179184 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314622 s, 33.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149641 s, 70.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284647 s, 36.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230576 s, 45.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212333 s, 49.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345777 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324589 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228821 s, 45.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386852 s, 27.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254484 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389155 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286011 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306822 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336891 s, 31.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.188684 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272376 s, 38.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062871 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059503 s, 17.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066896 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764062 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867241 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106889 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133741 s, 7.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12344 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127709 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122881 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116634 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113686 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125321 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141611 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112148 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321605 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235564 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335544 s, 31.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365591 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326269 s, 32.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310738 s, 33.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023461 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271449 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027028 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.046424 s, 22.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423172 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450462 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514014 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397335 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497088 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450848 s, 23.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=8 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 04:00:45 (1766394045) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.215674 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391134 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577989 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752214 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720612 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802513 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110158 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933288 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087616 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120387 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114986 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908302 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103923 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130234 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12512 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139307 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135431 s, 7.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357761 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146417 s, 71.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352779 s, 29.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00375102 s, 280 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106675 s, 98.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287 s, 36.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012909 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291912 s, 35.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412675 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377524 s, 27.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310098 s, 33.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376372 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387908 s, 27.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312548 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292409 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342712 s, 30.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.211261 s, 79.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703097 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071249 s, 14.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604761 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849553 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797886 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770456 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557815 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061798 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063639 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12141 s, 8.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143873 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159846 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17466 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134237 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176628 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170358 s, 6.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=5 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345529 s, 30.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326868 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245648 s, 42.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387297 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310025 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255493 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330446 s, 31.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330157 s, 31.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333508 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253084 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365992 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316996 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357011 s, 29.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305257 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328697 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552278 s, 19.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.177275 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332643 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428639 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917355 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118682 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126532 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146798 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160126 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146178 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175637 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145422 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168903 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168068 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162039 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181521 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166922 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169691 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168893 s, 62.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178579 s, 58.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00406053 s, 258 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109116 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161422 s, 65.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0467281 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00367421 s, 285 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179686 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490575 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00313183 s, 335 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540801 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022199 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168784 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.035692 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103447 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159218 s, 65.9 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (7s) == sanityn test 77e: check TBF NID nrs policy ============ 04:00:51 (1766394051) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.24@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.220084 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293902 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387237 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140405 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184184 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178068 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181299 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164363 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178602 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196508 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179979 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192451 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190612 s, 5.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199589 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184574 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18332 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201715 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161763 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160889 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218759 s, 47.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166496 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149345 s, 70.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234635 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134739 s, 77.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020774 s, 50.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810527 s, 129 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013406 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172274 s, 60.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142603 s, 73.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00432134 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719013 s, 146 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963671 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746909 s, 140 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.165356 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362386 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766183 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11303 s, 9.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128359 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123989 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143177 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112193 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118413 s, 8.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135744 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134639 s, 7.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127949 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136477 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141893 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144358 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166977 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190779 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179777 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165895 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212813 s, 49.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216282 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00373379 s, 281 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179443 s, 58.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294106 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261428 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247446 s, 42.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888164 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152724 s, 68.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314516 s, 33.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011626 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842883 s, 124 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621746 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108546 s, 96.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.164713 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534911 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0452145 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730534 s, 14.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128091 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122357 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126404 s, 8.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141449 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121594 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135937 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126274 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132056 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111336 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143661 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133147 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15311 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130245 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269714 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179383 s, 58.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150946 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178362 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183623 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207234 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199972 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285647 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296673 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305294 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249399 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315374 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302149 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299742 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299463 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600759 s, 175 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.209793 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478099 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0502586 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167652 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191777 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172868 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185886 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187655 s, 5.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185854 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208011 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209912 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210751 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20163 s, 5.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193717 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194484 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196138 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238903 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133451 s, 78.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252271 s, 41.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015648 s, 67.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186414 s, 56.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327586 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327584 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412694 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198874 s, 52.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200593 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033543 s, 31.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227084 s, 46.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267197 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00512207 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0049292 s, 213 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00466705 s, 225 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588333 s, 178 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 04:01:00 (1766394060) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.179317 s, 93.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346912 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356299 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1094 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817549 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129264 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130101 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140151 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146751 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155211 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144356 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141085 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156137 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131487 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132666 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148103 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138681 s, 7.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=3 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268585 s, 39.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167276 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167901 s, 62.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178108 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183765 s, 57.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229679 s, 45.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247024 s, 42.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258901 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206038 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284761 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025214 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262514 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292307 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259136 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284577 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314878 s, 33.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.20785 s, 80.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264659 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482456 s, 21.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0778813 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983311 s, 10.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131057 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12276 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119244 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107258 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137675 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132056 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12306 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139564 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144487 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138859 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14816 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140941 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895077 s, 117 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199065 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167159 s, 62.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175201 s, 59.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026445 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207552 s, 50.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202076 s, 51.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216641 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00372766 s, 281 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00398831 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105254 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247894 s, 42.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253781 s, 41.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309008 s, 33.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278792 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136999 s, 76.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.183068 s, 91.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301497 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597348 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0530805 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613428 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898405 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643831 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849018 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966406 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920253 s, 11.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128026 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114349 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130284 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126397 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139525 s, 7.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142116 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146404 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122921 s, 85.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148491 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168002 s, 62.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224692 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233532 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200188 s, 52.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217023 s, 48.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022017 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222054 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265265 s, 39.5 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321245 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347154 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341411 s, 30.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319862 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369921 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382196 s, 27.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.175976 s, 95.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038745 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449757 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118823 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103476 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11884 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107888 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127615 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140837 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12637 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132062 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113404 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151444 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132807 s, 7.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13403 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138757 s, 7.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149599 s, 7.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256901 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404394 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0043377 s, 242 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150591 s, 69.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019006 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225448 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173396 s, 60.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284742 s, 36.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264333 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305852 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558522 s, 188 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643519 s, 163 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229653 s, 45.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257505 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609726 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112773 s, 93.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (8s) == sanityn test 77g: Change TBF type directly ============ 04:01:08 (1766394068) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 04:01:12 (1766394072) oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg124-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg124-client: oleg124-server: ssh exited with exit code 2 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg124-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg124-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg124-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 PASS 77h (5s) == sanityn test 77i: Change rank of TBF rule ============= 04:01:18 (1766394078) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (10s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 04:01:27 (1766394087) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.174943 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357955 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363553 s, 28.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123898 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140743 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132341 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123631 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133343 s, 7.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127631 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151453 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151883 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159908 s, 6.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141402 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144093 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15155 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140509 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156611 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248346 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245961 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102282 s, 103 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236673 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011744 s, 89.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219943 s, 47.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227721 s, 46.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206953 s, 50.7 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345709 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312725 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243922 s, 43.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100143 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00428099 s, 245 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00409531 s, 256 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312903 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364862 s, 28.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=6 removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93023 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4481 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (42s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 04:02:09 (1766394129) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.151366 s, 111 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442305 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657052 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078067 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730887 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596448 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786222 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835969 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928651 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774121 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891911 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755156 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121927 s, 8.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135233 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14608 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11357 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142568 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209038 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213949 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176536 s, 59.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235463 s, 44.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239469 s, 43.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238443 s, 44.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256398 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206332 s, 50.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251411 s, 41.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200508 s, 52.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022997 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02848 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271484 s, 38.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222554 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261082 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286776 s, 36.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6546 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9625 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1515 s, 111 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0858947 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129723 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119219 s, 8.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11442 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132896 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12252 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14073 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113602 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129162 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140974 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127145 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13226 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161724 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139854 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162476 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164484 s, 6.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256243 s, 40.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247971 s, 42.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197539 s, 53.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261478 s, 40.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271086 s, 38.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286436 s, 36.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236047 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027439 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199952 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155105 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207258 s, 50.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285701 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284438 s, 36.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273779 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00457372 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582478 s, 180 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6465 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9565 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (114s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 04:04:03 (1766394243) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.146735 s, 114 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796867 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107445 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946946 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114289 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125154 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12198 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113452 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133959 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126337 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145255 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135152 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124556 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125541 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147814 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145265 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139729 s, 7.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017668 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202909 s, 51.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192708 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209115 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191639 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166806 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146095 s, 71.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218629 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104386 s, 100 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253321 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770669 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00514116 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0040587 s, 258 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252667 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610352 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0020681 s, 507 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8306 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9656 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg124-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.144513 s, 116 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537417 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0959778 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106115 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752479 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108025 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861023 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112975 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103523 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10438 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110233 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109146 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11978 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105752 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107072 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102988 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140963 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177272 s, 59.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132658 s, 79.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187663 s, 55.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019223 s, 54.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203132 s, 51.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241903 s, 43.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170519 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205461 s, 51.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246397 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163854 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204457 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00353205 s, 297 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639974 s, 164 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00376522 s, 278 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268056 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00520791 s, 201 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8299 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.961 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (112s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 04:05:55 (1766394355) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136632 s, 123 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388933 s, 27.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140094 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140279 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129238 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150224 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157775 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144886 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139164 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107485 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149034 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160676 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157199 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149437 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126548 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142334 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173505 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117881 s, 89.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283474 s, 37.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228019 s, 46.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198476 s, 52.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131164 s, 79.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244289 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150413 s, 69.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983111 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161391 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216211 s, 48.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666251 s, 157 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010232 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234288 s, 44.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01415 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133412 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121625 s, 86.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89968 s, 21.4 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73632 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.24@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.24@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.153641 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348866 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392144 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0495492 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981824 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0886567 s, 11.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111361 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122318 s, 8.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13219 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136382 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118468 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100556 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117083 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129637 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106399 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114738 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111049 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963858 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992186 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159139 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116242 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013727 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0029645 s, 354 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148499 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290506 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394184 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196334 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240902 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710842 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235949 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309288 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00213357 s, 491 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00231517 s, 453 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90644 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7395 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.24@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.182038 s, 92.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243584 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353792 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077322 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104943 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100194 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119229 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124052 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115706 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11048 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111684 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098414 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127781 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127489 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113864 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136439 s, 7.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106063 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128145 s, 81.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187055 s, 56.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169744 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147743 s, 71.0 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284025 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249067 s, 42.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288765 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168733 s, 62.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224141 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219259 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266437 s, 39.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086185 s, 122 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108043 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027975 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00507739 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00476204 s, 220 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=2 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9142 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89788 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.179729 s, 93.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374381 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050856 s, 20.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163355 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178732 s, 5.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189449 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164663 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197214 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206162 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201676 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215794 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207748 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175735 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194671 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201965 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206031 s, 5.1 MB/s 1+0 records in 1+0 records out running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206367 s, 5.1 MB/s execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191888 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195627 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209234 s, 50.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173771 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230713 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252523 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251689 s, 41.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023667 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271241 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221974 s, 47.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422537 s, 24.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250032 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266188 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269578 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348394 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350687 s, 29.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77248 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75151 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9027 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74116 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.190553 s, 88.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262602 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295523 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836591 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681884 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140555 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130186 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139913 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144953 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142555 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133473 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128565 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142293 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153634 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146482 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139474 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148231 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176379 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252439 s, 41.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200483 s, 52.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030062 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023916 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172653 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291121 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248317 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242983 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240794 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308488 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030022 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921913 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138648 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355826 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970706 s, 108 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4593 s, 5.4 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4373 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.14563 s, 115 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362882 s, 28.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970432 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100121 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0984379 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874463 s, 12.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132917 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138405 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190596 s, 55.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181344 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169262 s, 61.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383124 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388556 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289134 s, 36.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195024 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018636 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161507 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158549 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180353 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203485 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167458 s, 62.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306071 s, 34.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202235 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184887 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208494 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032604 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206724 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203107 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404782 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20972 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004166 s, 252 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00486025 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0041897 s, 250 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91408 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73955 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.157768 s, 106 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261836 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660118 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988799 s, 10.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113609 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109381 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10902 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1059 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100918 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797603 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162834 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259692 s, 40.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406734 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202414 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182513 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168455 s, 62.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0451342 s, 23.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356949 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00425995 s, 246 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18581 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123331 s, 85.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357627 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205692 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149214 s, 70.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395685 s, 26.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286091 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188685 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192657 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00389309 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198919 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20274 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194028 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199458 s, 5.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=9 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75331 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7356 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75616 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.743 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (267s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 04:10:22 (1766394622) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.131747 s, 127 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362078 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319346 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885343 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092165 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104218 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131511 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127268 s, 8.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13542 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130124 s, 8.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13104 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135915 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142483 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143778 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138735 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158805 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142061 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139929 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123908 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236925 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155778 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150754 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176675 s, 59.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026061 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173813 s, 60.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181933 s, 57.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179277 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208096 s, 50.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201117 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0034816 s, 301 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236121 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256762 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246401 s, 42.6 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90429 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73586 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn Waiting for MDT destroys to complete PASS 77kb (26s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 04:10:48 (1766394648) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.24@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.24@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.151626 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322204 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361651 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0866215 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101814 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918145 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889168 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097255 s, 10.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120272 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105757 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117789 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11323 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127572 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12624 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10417 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127198 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105469 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10297 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110428 s, 9.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106115 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979235 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101162 s, 10.4 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116551 s, 9.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109479 s, 9.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108485 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107389 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105735 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116395 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106124 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111991 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115526 s, 9.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110607 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111145 s, 9.4 MB/s removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90829 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74351 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn Waiting for MDT destroys to complete PASS 77kc (30s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 04:11:18 (1766394678) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.24@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.170161 s, 98.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036225 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308429 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560662 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0728395 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078806 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723736 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0839474 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874657 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717359 s, 14.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104551 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114708 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134725 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140139 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125541 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136774 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140954 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134256 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246512 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022892 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185018 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200867 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205163 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238182 s, 44.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216981 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024862 s, 42.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265519 s, 39.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297835 s, 35.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295989 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270691 s, 38.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284027 s, 36.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301714 s, 34.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310561 s, 33.8 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92834 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.99884 s, 21.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn Waiting for MDT destroys to complete PASS 77kd (21s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 04:11:39 (1766394699) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.149365 s, 112 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279649 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033308 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0501516 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102853 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111818 s, 9.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119045 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125411 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129564 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120159 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116071 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128972 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136711 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142717 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132288 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139876 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126539 s, 8.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162015 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139203 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205068 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193054 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118671 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198812 s, 52.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175222 s, 59.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967664 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937718 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139983 s, 74.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239485 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116154 s, 90.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252232 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026347 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841255 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00302188 s, 347 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=4 removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76847 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75439 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92882 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74592 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn Waiting for MDT destroys to complete PASS 77ke (56s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 04:12:35 (1766394755) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.157238 s, 107 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436364 s, 24.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751702 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969768 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945921 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104905 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1354 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116176 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111006 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138736 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150328 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150819 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123772 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1418 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120686 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14989 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138011 s, 7.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140937 s, 74.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166308 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136707 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184562 s, 56.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021183 s, 49.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185868 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190804 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139127 s, 75.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190357 s, 55.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205332 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201455 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210372 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265265 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244738 s, 42.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00398434 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00463145 s, 226 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6447 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9586 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn Waiting for MDT destroys to complete PASS 77kf (52s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 04:13:27 (1766394807) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.172551 s, 97.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170219 s, 6.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168242 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197892 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191467 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198266 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203995 s, 5.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185364 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19145 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184167 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199341 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205643 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179983 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184581 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199409 s, 5.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191635 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189415 s, 5.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217574 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962938 s, 109 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00393649 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0053724 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00465042 s, 225 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415778 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348497 s, 30.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280265 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00336834 s, 311 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310498 s, 33.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193208 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274031 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906347 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441787 s, 23.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208894 s, 50.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197186 s, 53.2 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91222 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74269 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg124-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.184144 s, 91.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295936 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511394 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843027 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10398 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105121 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182261 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183057 s, 5.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166888 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173853 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174549 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182008 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168814 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17449 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187341 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191362 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199207 s, 5.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161902 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210913 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174001 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174596 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209448 s, 50.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277128 s, 37.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968689 s, 108 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280764 s, 37.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013789 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374549 s, 28.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309976 s, 33.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187823 s, 55.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330703 s, 31.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117134 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196158 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00382875 s, 274 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76497 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74077 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75214 s, 10.8 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73994 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete PASS 77kg (86s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 04:14:53 (1766394893) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 04:14:57 (1766394897) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg124-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.19943 s, 328 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg124-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0078 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 04:15:47 (1766394947) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.165453 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254547 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313095 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106472 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117979 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114069 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129804 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134601 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131075 s, 8.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125642 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127136 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137256 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124703 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123826 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124877 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109671 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11573 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212847 s, 49.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233709 s, 44.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178654 s, 58.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025388 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247124 s, 42.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215157 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233805 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022905 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325159 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237538 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337801 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367858 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029615 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455622 s, 23.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333403 s, 31.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319172 s, 32.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=1 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90268 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97764 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1715 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253645 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429941 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449337 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610074 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701853 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791463 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843096 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832506 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103438 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091918 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115059 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118303 s, 8.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9f6fd06fa000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111267 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117583 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118466 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13837 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811017 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830623 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802744 s, 13.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903155 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991527 s, 10.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091937 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09908 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088665 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861856 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912835 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0950493 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089175 s, 11.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0859366 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901292 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094336 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913612 s, 11.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9f6fd06fa000.lock_unused_count=4 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg124-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg124-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90138 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97274 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (48s) == sanityn test 77o: Changing rank should not panic ====== 04:16:35 (1766394995) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 04:16:39 (1766394999) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (40s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 04:17:19 (1766395039) mds.MDS.mdt.nrs_policies=tbf oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg124-client: oleg124-server: ssh exited with exit code 36 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 oleg124-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg124-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 04:17:32 (1766395052) striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d77r.sanityn mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 339348 kill md_thread xargs PASS 77r (39s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 04:18:11 (1766395091) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 04:18:15 (1766395095) striped dir -i1 -c2 -H crush2 /mnt/lustre/d79.sanityn File: /mnt/lustre/d79.sanityn Size: 8192 Blocks: 16 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798763914 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:18:15.000000000 -0500 Modify: 2025-12-22 04:18:15.000000000 -0500 Change: 2025-12-22 04:18:15.000000000 -0500 Birth: 2025-12-22 04:18:16.000000000 -0500 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 04:18:28 (1766395108) total: 10 open/close in 0.04 seconds: 269.63 ops/second multiop /mnt/lustre2/d80a.sanityn/f80a.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Finish migration, then checking.. PASS 80a (4s) == sanityn test 80b: Accessing directory during migration ========================================================== 04:18:31 (1766395111) start migration thread 347755 accessing the migrating directory for 1 minute... concurrent access 1660 failures, 2372 successes /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 347755 Killed ( while true; do mdt_idx=$((RANDOM % MDSCOUNT)); $LFS migrate -m $mdt_idx $migdir1 &> /dev/null; done ) (wd: ~) PASS 80b (62s) == sanityn test 81a: rename and stat under striped directory ========================================================== 04:19:33 (1766395173) stat: cannot statx 'd0/0': No such file or directory File: d0/0 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289284105 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:19:34.000000000 -0500 Modify: 2025-12-22 04:19:34.000000000 -0500 Change: 2025-12-22 04:19:34.000000000 -0500 Birth: 2025-12-22 04:19:35.000000000 -0500 PASS 81a (3s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 04:19:36 (1766395176) total: 2500 open/close in 8.20 seconds: 304.71 ops/second /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 353741 Killed ( while true; do touch $DIR1/$tdir; done ) (wd: ~) PASS 81b (53s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 04:20:29 (1766395229) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 04:20:30 (1766395230) total: 2500 open/close in 6.70 seconds: 373.27 ops/second total: 2500 open/close in 5.73 seconds: 436.02 ops/second snapshot_time 1766395273.502955759 secs.nsecs start_time 1766395232.093289853 secs.nsecs elapsed_time 41.409665906 secs.nsecs open 2505 samples [usecs] 87 4375 976009 428241643 close 2504 samples [usecs] 46 269 243110 26395062 mknod 2501 samples [usecs] 153 4287 835111 319420019 unlink 2500 samples [usecs] 203 20314 1500289 2917558305 mkdir 2 samples [usecs] 854 2018 2872 4801640 rmdir 2 samples [usecs] 1679 2870 4549 11055941 rename 2500 samples [usecs] 197 5717 3154437 6108391939 getattr 3764 samples [usecs] 0 5988 122337 52187577 statfs 8 samples [usecs] 12 20 126 2054 parallel_rename_file 2500 samples [usecs] 197 5717 3154437 6108391939 crossdir_rename 2500 samples [usecs] 197 5717 3154437 6108391939 snapshot_time 1766395273.503055772 secs.nsecs start_time 1766395232.093339493 secs.nsecs elapsed_time 41.409716279 secs.nsecs open 2504 samples [usecs] 70 1262 835392 303943406 close 2504 samples [usecs] 46 2507 216743 26690011 mknod 2500 samples [usecs] 145 1099 711303 221823663 unlink 2500 samples [usecs] 199 18819 1512278 2905995688 mkdir 2 samples [usecs] 832 6244 7076 39679760 rmdir 2 samples [usecs] 2247 5452 7699 34773313 rename 2500 samples [usecs] 196 5729 3013298 5342379098 getattr 3757 samples [usecs] 7 16560 127990 297555988 statfs 8 samples [usecs] 6 95 183 10239 parallel_rename_file 2500 samples [usecs] 196 5729 3013298 5342379098 crossdir_rename 2500 samples [usecs] 196 5729 3013298 5342379098 PASS 81d (44s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 04:21:14 (1766395274) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.8201 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.8201 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 04:21:17 (1766395277) start pid 368150 to create/unlink striped directory start pid 368151 to stat striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5797: 368150 Terminated ( cd $DIR1; while true; do $LFS mkdir -i1 -c2 $tdir; rmdir $tdir; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5797: 368151 Terminated ( cd $DIR2; while true; do stat $tdir > /dev/null 2>&1; done ) PASS 83 (122s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 04:23:19 (1766395399) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 85: Lustre API root cache race =========== 04:23:27 (1766395407) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1766395407 Finishing test test1 at 1766395408 PASS 85 (2s) == sanityn test 90: open/create and unlink striped directory ========================================================== 04:23:29 (1766395409) start pid 378778 to open/create under striped directory start pid 378779 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5864: 378778 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; touch $tdir/f{0..3} > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5864: 378779 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 90 (183s) == sanityn test 91: chmod and unlink striped directory === 04:26:32 (1766395592) start pid 391286 to chmod striped directory start pid 391287 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5901: 391286 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; chmod go+w $tdir > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5901: 391287 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 91 (182s) == sanityn test 92: create remote directory under orphan directory ========================================================== 04:29:34 (1766395774) fail_loc=0x1408 mkdir: cannot create directory '/mnt/lustre2/d92.sanityn/dir': No such file or directory lfs setdirstripe: cannot resolve path '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory (2) lfs setdirstripe: '/mnt/lustre2/d92.sanityn/remote_dir' is not on a Lustre filesystem: No such file or directory (2) lfs setdirstripe: cannot create dir '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory fail_loc=0 PASS 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 04:29:36 (1766395776) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5270 0x1496 0x2c0000400 0 5283 0x14a3 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6467 0x1943 0x280000401 1 6451 0x1933 0x2c0000401 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 04:29:45 (1766395785) osc.lustre-OST0000-osc-ffff9f6fd06fa000.idle_timeout=0 osc.lustre-OST0000-osc-ffff9f6fe073e000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9f6fd06fa000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9f6fe073e000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0121996 s, 671 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09715 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5988: 410027 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00591416 s, 693 kB/s osc.lustre-OST0000-osc-ffff9f6fd06fa000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9f6fe073e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6fd06fa000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6fe073e000.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 04:29:55 (1766395795) llite.lustre-ffff9f6fd06fa000.fast_read=0 llite.lustre-ffff9f6fe073e000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0185088 s, 113 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff9f6fd06fa000.fast_read=1 llite.lustre-ffff9f6fe073e000.fast_read=1 PASS 95a (15s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 04:30:10 (1766395810) llite.lustre-ffff9f6fd06fa000.fast_read=0 llite.lustre-ffff9f6fe073e000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00192867 s, 6.4 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff9f6fd06fa000.fast_read=1 llite.lustre-ffff9f6fe073e000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 04:30:18 (1766395818) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 04:30:20 (1766395820) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00151796 s, 2.7 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 04:30:22 (1766395822) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0246361 s, 85.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 04:30:25 (1766395825) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0232027 s, 90.4 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 04:30:27 (1766395827) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00291195 s, 4.1 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 04:30:29 (1766395829) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000624137 s, 6.6 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 04:30:32 (1766395832) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000545097 s, 7.5 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 04:30:34 (1766395834) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000554591 s, 7.4 MB/s PASS 101c (3s) == sanityn test 102: Test open by handle of unlinked file ========================================================== 04:30:37 (1766395837) file: /mnt/lustre/f102.sanityn fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 00 02 00 00 00 62 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x200000402:0x3262:0x0] access via mount point '/mnt/lustre2' - OK access after unlink - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.remote fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 40 02 00 00 00 f0 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000402:0x3af0:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.remote/subdir fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 00 02 00 00 00 63 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x200000402:0x3263:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.stripe fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 40 02 00 00 00 f1 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000402:0x3af1:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.stripe/subdir fh_bytes: 32 fh_type: 151 fh_data: 02 04 00 40 02 00 00 00 f2 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000402:0x3af2:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! PASS 102 (3s) == sanityn test 103: Test size correctness with lockahead ========================================================== 04:30:40 (1766395840) striped dir -i1 -c2 -H crush2 /mnt/lustre/d103.sanityn fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1766395841 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1766395844 Finishing test test23 at 1766395844 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 04:30:48 (1766395848) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00305563 s, 335 kB/s oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395849 1766395849 1766395849 MDT a|m|ctime 1766395849 1766395849 1766395849 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00297075 s, 345 kB/s oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395849 1766395852 1766395852 MDT a|m|ctime 1766395849 1766395852 1766395852 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00806128 s, 127 kB/s oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395849 1766395856 1766395856 MDT a|m|ctime 1766395849 1766395856 1766395856 oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395849 1766395856 1766395856 MDT a|m|ctime 1766395849 1766395856 1766395856 oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1766395849 1766395856 1766395865 MDT a|m|ctime 1766395849 1766395856 1766395865 PASS 104 (20s) == sanityn test 105: Glimpse and lock cancel race ======== 04:31:08 (1766395868) striped dir -i1 -c2 -H crush2 /mnt/lustre/d105.sanityn fail_loc=0x416 PASS 105 (23s) == sanityn test 106a: Verify the btime via statx() ======= 04:31:31 (1766395891) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00285845 s, 358 kB/s oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 5142 Type: regular Mode: 0644 Flags: 0x80000 Generation: 3231702604 Version: 0x00000002:000176ae User: 0 Group: 0 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69490ff3:00000000 -- Mon Dec 22 04:31:31 2025 atime: 0x69490ff3:00000000 -- Mon Dec 22 04:31:31 2025 mtime: 0x69490ff3:00000000 -- Mon Dec 22 04:31:31 2025 crtime: 0x69490ff4:1ff2b600 -- Mon Dec 22 04:31:32 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200000402:0x3267:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 67 32 00 00 00 00 00 00 02 04 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 c0 02 00 00 00 38 19 00 00 00 00 00 00 00 00 00 00 01 00 00 00 user.job (4) = "dd.0" linkea: idx=0 parent=[0x200000007:0x1:0x0] name='f106a.sanityn' trusted.som (24) = 04 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EXTENTS: PASS 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 04:31:33 (1766395893) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00410477 s, 249 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 8 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289292392 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:31:34.000000000 -0500 Modify: 2025-12-22 04:31:34.000000000 -0500 Change: 2025-12-22 04:31:34.000000000 -0500 Birth: 2025-12-22 04:31:34.000000000 -0500 /mnt/lustre/f106b.sanityn 144115205289292392 -rw-r--r-- 1 0 0 1766395894 1766395894 1766395894 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 8 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289292392 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:31:34.000000000 -0500 Modify: 2025-12-22 04:31:34.000000000 -0500 Change: 2025-12-22 04:31:34.000000000 -0500 Birth: 2025-12-22 04:31:34.000000000 -0500 1766395894 1024 8 PASS 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 04:31:35 (1766395895) PASS 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 04:31:38 (1766395898) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.100862 s, 104 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6475 0x194b 0x280000401 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 04:31:42 (1766395902) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.107623 s, 97.4 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6458 0x193a 0x2c0000401 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.8201 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 04:31:50 (1766395910) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00185677 s, 5.4 MB/s llite.lustre-ffff9f6fd06fa000.hybrid_io=0 llite.lustre-ffff9f6fe073e000.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.20431 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 04:31:57 (1766395917) 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 192.168.201.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.124@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 PASS 109 (623s) == sanityn test 110: do not grant another lock on resend ========================================================== 04:42:20 (1766396540) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934275 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:42:21.000000000 -0500 Modify: 2025-12-22 04:42:21.000000000 -0500 Change: 2025-12-22 04:42:21.000000000 -0500 Birth: 2025-12-22 04:42:22.000000000 -0500 fail_loc=0 fail_val=0 Connected clients: oleg124-client.virtnet oleg124-client.virtnet PASS 110 (67s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 04:43:27 (1766396607) fail_loc=0x8000018a /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6784: 482770 Terminated $MULTIOP $DIR2/$tdir/mdt0dir/foodir/file2 Ow4096_c PASS 111 (9s) == sanityn test 112: update max-inherit in default LMV === 04:43:36 (1766396616) lmv.lustre-clilmv-ffff9f6fc6324800.qos_threshold_rr=90 lmv.lustre-clilmv-ffff9f6fe0123000.qos_threshold_rr=90 PASS 112 (8s) == sanityn test 113: check servers of specified fs ======= 04:43:44 (1766396624) lustre-OST0000-osc-ffff9f6fe0123000 active. lustre-OST0001-osc-ffff9f6fe0123000 active. lustre-OST0000-osc-ffff9f6fe0123000 active. lustre-OST0001-osc-ffff9f6fe0123000 active. MGC192.168.201.124@tcp active. PASS 113 (4s) == sanityn test 114: implicit default LMV inherit ======== 04:43:49 (1766396629) /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1 lmv_stripe_count: 2 lmv_stripe_offset: 2 lmv_hash_type: none lmv_max_inherit: 2 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1 lmv_stripe_count: 2 lmv_stripe_offset: 2 lmv_hash_type: none lmv_max_inherit: 2 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 PASS 114 (13s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 04:44:01 (1766396641) - create 2069 (time 1766396653.43 total 10.00 last 206.87) total: 3000 create in 14.43 seconds: 207.83 ops/second oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: htree_dump_int_block: Ext2 directory block not found while calculating checksum for htree root oleg124-server: USE: f1389 f1672 fail_loc=0x80002401 fail_val=5 mkdir: cannot create directory '/mnt/lustre/d115.sanityn/f1672': File exists PASS 115 (27s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 04:44:28 (1766396668) llite.lustre-ffff9f6fc6324800.intent_mkdir=1 llite.lustre-ffff9f6fe0123000.intent_mkdir=1 MD layout /mnt/lustre/d116.sanityn: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x7d0:0x0] 1 [0x240000404:0x75f:0x0] mkdir /mnt/lustre/d116.sanityn/tdir0 setdirstripe -D -i1 /mnt/lustre2/d116.sanityn/tdir0 mkdir /mnt/lustre/d116.sanityn/tdir0/tdir11 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none mdtidx FID[seq:oid:ver] lmv_stripe_count: 0 lmv_stripe_offset: 1 lmv_hash_type: none mdtidx FID[seq:oid:ver] PASS 116 (4s) == sanityn test 117a: TCU: Init and enable Trash Can on MDTs ========================================================== 04:44:33 (1766396673) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 PASS 117a (6s) == sanityn test 117b: Move regular file and empty dir into trash can dir ========================================================== 04:44:39 (1766396679) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain llite.lustre-ffff9f6fc6324800.trash_file_read=1 llite.lustre-ffff9f6fe0123000.trash_file_read=1 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.114808 s, 36.5 MB/s /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0x200000bd1:0xbcc:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/trash/MDT0001: llite.lustre-ffff9f6fc6324800.trash_file_read=0 llite.lustre-ffff9f6fe0123000.trash_file_read=0 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117b (8s) == sanityn test 117c: Move deleted tree with multiple levels into trash ========================================================== 04:44:47 (1766396687) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: tdir1 tdir2 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir1: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir1/tdir11: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir1/tdir12: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0/tdir2: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117c (8s) == sanityn test 117d: Per-User Trash can Type testing ==== 04:44:55 (1766396695) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2/tdir21] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2/tdir22] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117d.sanityn/tdir2/tfile23] uid1: 0 uid2: 500 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: 0x200000bd1:0xbdb:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200000bd1:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/trash/MDT0000/0/0x200000bd1:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/trash/MDT0000/0/0x200000bd1:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: 0x200000bd1:0xbdf:0x0 /mnt/lustre/.lustre/trash/MDT0000/500/0x200000bd1:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/trash/MDT0000/500/0x200000bd1:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/trash/MDT0000/500/0x200000bd1:0xbdf:0x0/tdir22: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117d (7s) == sanityn test 117e: Undeleted dir in trash should keep its original xattrs ========================================================== 04:45:02 (1766396702) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/trash/MDT0000/500: trusted.name11="value11" trusted.name12="value12" trusted.name1="value1" mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117e (7s) == sanityn test 117f: Uncache the dentry under the trash dir ========================================================== 04:45:09 (1766396709) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/500: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: tdir1 tdir2 tfile3 tfile4 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir2: /mnt/lustre/.lustre/trash/MDT0000/500: tdir2 tfile4 -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0 /mnt/lustre/.lustre/trash/MDT0000/0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/500 /mnt/lustre/.lustre/trash/MDT0000/500: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir2 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir2: -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tdir1: -u------------e------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tfile4 -u------------e------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0/tfile3 -u-------------------- /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117f (7s) == sanityn test 117g: Access .Trash for a non-striped directory ========================================================== 04:45:17 (1766396717) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/500: ==> check '.Trash': tdir1 tfile3 <== tdir1 tfile3 tdir1 tfile3 tdir21 tfile22 tdir2 tdir21 tfile22 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117g (8s) == sanityn test 117h: Permission testing of various operations for files in Trash Can ========================================================== 04:45:25 (1766396725) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Trash list: ===> /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/tdir1: /mnt/lustre/.lustre/trash/MDT0000/500: <=== touch: setting times of '/mnt/lustre/.lustre/trash/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/trash/f117h.sanityn.crfail': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.crfail': Operation not permitted rmdir: failed to remove '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/f117h.sanityn.crfail': No such file or directory mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0/d117h.sanityn.mvfail': Operation not permitted chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3 chmod: changing permissions of '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3': Bad file descriptor truncate: cannot truncate '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3' to length 4096: Bad file descriptor chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1 chmod: changing permissions of '/mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1': Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0//tdir1: Bad file descriptor open: Bad file descriptor open: Bad file descriptor read: Operation not permitted mmap: Operation not permitted open: Is a directory open: Is a directory mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117h (8s) == sanityn test 117i: Forbid to create a file with name .Trash ========================================================== 04:45:33 (1766396733) mkdir: cannot create directory '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument touch: setting times of '/mnt/lustre/d117i.sanityn/.Trash': No such file or directory open(O_RDWR|O_CREAT): Invalid argument mcreate: cannot create `/mnt/lustre/d117i.sanityn/.Trash' with mode 0100644: Invalid argument ln: failed to create hard link '/mnt/lustre/d117i.sanityn/.Trash' => '/mnt/lustre/d117i.sanityn/f117i.sanityn': Invalid argument ln: failed to create symbolic link '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument mv: cannot move '/mnt/lustre/d117i.sanityn/f117i.sanityn' to a subdirectory of itself, '/mnt/lustre/d117i.sanityn/.Trash' mv: cannot move '/mnt/lustre/d117i.sanityn/f117i.sanityn' to a subdirectory of itself, '/mnt/lustre/.Trash' ln: failed to create symbolic link '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument mv: cannot move '/mnt/lustre/d117i.sanityn/d117i.sanityn' to a subdirectory of itself, '/mnt/lustre/d117i.sanityn/.Trash' mv: cannot move '/mnt/lustre/d117i.sanityn/d117i.sanityn' to a subdirectory of itself, '/mnt/lustre/.Trash' PASS 117i (3s) == sanityn test 117j: Move back file from Trash Can ====== 04:45:36 (1766396736) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain tfile.1 -u------------e------- /mnt/lustre/d117j.sanityn/.Trash/tfile.1 --------------e------- /mnt/lustre/d117j.sanityn/tfile.1 List /mnt/lustre/d117j.sanityn/.Trash /mnt/lustre/d117j.sanityn/.Trash: tdir1 /mnt/lustre/d117j.sanityn/.Trash/tdir1: tdir11 tfile12 tfile13 /mnt/lustre/d117j.sanityn/.Trash/tdir1/tdir11: -u-------------------- /mnt/lustre/d117j.sanityn/.Trash/tdir1 /mnt/lustre/d117j.sanityn/.Trash/tdir1: -u-------------------- /mnt/lustre/d117j.sanityn/.Trash/tdir1/tdir11 /mnt/lustre/d117j.sanityn/.Trash/tdir1/tdir11: -u------------e------- /mnt/lustre/d117j.sanityn/.Trash/tdir1/tfile12 -u------------e------- /mnt/lustre/d117j.sanityn/.Trash/tdir1/tfile13 === DONE === mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117j (7s) == sanityn test 117k: Unrm file from Trash Can =========== 04:45:43 (1766396743) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain ==== DONE ==== mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117k (9s) == sanityn test 117l: Clean up files in Trash Can via command 'lfs trash clean' ========================================================== 04:45:52 (1766396752) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Phase 0/PR: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 0/PR: remove dir 1:0 from Trash Can Phase 2/PO: remove dir 1:0 from Trash Can Phase 1/EN: remove dir 1:0 from Trash Can Phase 0/PR: remove dir 1:500 from Trash Can Phase 2/PO: remove dir 1:500 from Trash Can Phase 1/EN: remove dir 1:500 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc06:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc0c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbd2:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 2:tdir1 from Trash Can Phase 0/PR: remove dir 3:tdir11 from Trash Can Phase 2/PO: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove file 3:tfile12 from Trash Can Phase 1/EN: remove file 3:tfile13 from Trash Can Phase 2/PO: remove dir 2:tdir1 from Trash Can Phase 1/EN: remove dir 2:tdir1 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc12:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe7:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbcc:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbec:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbf3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 0/PR: remove dir 1:0 from Trash Can Phase 2/PO: remove dir 1:0 from Trash Can Phase 1/EN: remove dir 1:0 from Trash Can Phase 0/PR: remove dir 1:500 from Trash Can Phase 2/PO: remove dir 1:500 from Trash Can Phase 1/EN: remove dir 1:500 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc06:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc0c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbd2:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 2:tdir1 from Trash Can Phase 0/PR: remove dir 3:tdir11 from Trash Can Phase 2/PO: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove file 3:tfile12 from Trash Can Phase 1/EN: remove file 3:tfile13 from Trash Can Phase 2/PO: remove dir 2:tdir1 from Trash Can Phase 1/EN: remove dir 2:tdir1 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc12:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe7:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbcc:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbec:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbf3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Phase 1/EN: remove file 2:tfile22 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc1c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117l (7s) == sanityn test 117m: Empty Trash Can via the command 'lfs trash clean' ========================================================== 04:45:59 (1766396759) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: tdir1 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0/tdir1: tdir11 tfile12 tfile13 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0/tdir1/tdir11: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: tfile22 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: Phase 0/PR: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe3:0x0 from Trash Can Phase 0/PR: remove dir 1:0 from Trash Can Phase 2/PO: remove dir 1:0 from Trash Can Phase 1/EN: remove dir 1:0 from Trash Can Phase 0/PR: remove dir 1:500 from Trash Can Phase 2/PO: remove dir 1:500 from Trash Can Phase 1/EN: remove dir 1:500 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc06:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc06:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc0c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc0c:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbd2:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbd2:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc12:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc12:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc25:0x0 from Trash Can Phase 1/EN: remove file 2:tfile22 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc25:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc25:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc25:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc1f:0x0 from Trash Can Phase 0/PR: remove dir 2:tdir1 from Trash Can Phase 0/PR: remove dir 3:tdir11 from Trash Can Phase 2/PO: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove dir 3:tdir11 from Trash Can Phase 1/EN: remove file 3:tfile12 from Trash Can Phase 1/EN: remove file 3:tfile13 from Trash Can Phase 2/PO: remove dir 2:tdir1 from Trash Can Phase 1/EN: remove dir 2:tdir1 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc1f:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc1f:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc1f:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbe7:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbe7:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbcc:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbcc:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbec:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbec:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xbf3:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xbf3:0x0 from Trash Can Phase 0/PR: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Phase 2/PO: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can Skip stub dir 1:0x200000bd1:0xc1c:0x0 Phase 1/EN: remove dir 1:0x200000bd1:0xc1c:0x0 from Trash Can mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117m (9s) == sanityn test 117n: Update LinkEA accordingly when moving file into Trash Can ========================================================== 04:46:08 (1766396768) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tfile.1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tfile.1' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tdir11: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tdir11' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile12: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile12' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1/tfile13: count 1 0: pfid [0x200000bd0:0x7eb:0x0], name 'tfile13' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/.Trash//tdir1: count 1 0: pfid [0x200000bd0:0x7ea:0x0], name 'tdir1' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x7ec:0x0], name 'tfile22' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x7ec:0x0], name 'tfile22' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x7ec:0x0], name 'tfile22' /mnt/lustre/d117n.sanityn/tdir2/.Trash//tfile22: count 1 0: pfid [0x200000bd0:0x7ec:0x0], name 'tfile22' mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117n (8s) == sanityn test 117o: Set/Clear NOTRASH flag on file ===== 04:46:16 (1766396776) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Set flags 0X80000/0X80040 OK: rc=0 file: /mnt/lustre/d117o.sanityn/tfile.1, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tfile.1, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1/tdir11, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1/tdir11, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1/tfile12, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 lsattr /mnt/lustre/d117o.sanityn/tdir1: ------d--------------- /mnt/lustre/d117o.sanityn/tdir1/tdir11 ------d-------e------- /mnt/lustre/d117o.sanityn/tdir1/tfile12 lsattr /mnt/lustre/d117o.sanityn/tdir1/tdir11: lsattr /mnt/lustre/d117o.sanityn/tdir1/tfile12: ------d-------e------- /mnt/lustre/d117o.sanityn/tdir1/tfile12 file: /mnt/lustre/d117o.sanityn/tdir1/tdir11, flags: 0X40 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1/tfile12, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 file: /mnt/lustre/d117o.sanityn/tdir1/tfile13, flags: 0X80040 notrash, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 Set flags 0X80040/0X80000 OK: rc=0 chattr 1.47.3-wc2 (11-Nov-2025) Flags of /mnt/lustre/d117o.sanityn/tdir1 set as ---------------------- file: /mnt/lustre/d117o.sanityn/tdir1/tfile13, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396780 tfile13 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117o (9s) == sanityn test 117p: Trash Can handling for remote dir/file ========================================================== 04:46:25 (1766396785) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/d117p.sanityn/d117p.sanityn.1/.Trash/f117p.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6499 0x1963 0x280000401 /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: f117p.sanityn /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: d117p.sanityn.01 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0/d117p.sanityn.01: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: f117p.sanityn mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117p (8s) == sanityn test 117pb: Trash Can handling for remote dir/file with multiple levels ========================================================== 04:46:33 (1766396793) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: d117pb.sanityn.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0/d117pb.sanityn.0: d117pb.sanityn.01 f117pb.sanityn.02 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0/d117pb.sanityn.0/d117pb.sanityn.01: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117pb (7s) == sanityn test 117q: Access Trash Can via '.Trash' for striped directory ========================================================== 04:46:40 (1766396800) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain #0 -> 0x200000bd0:0x7ef:0x0 #1 -> 0x240000404:0x760:0x0 file.1 file.2 Creating 100 files under striped directory: /mnt/lustre/d117q.sanityn/striped_dir total: 100 open/close in 0.69 seconds: 145.57 ops/second f117q.sanityn0 -> 0 f117q.sanityn1 -> 0 f117q.sanityn2 -> 0 f117q.sanityn3 -> 0 f117q.sanityn4 -> 0 f117q.sanityn5 -> 0 f117q.sanityn6 -> 0 f117q.sanityn7 -> 0 f117q.sanityn8 -> 0 f117q.sanityn9 -> 0 f117q.sanityn10 -> 0 f117q.sanityn11 -> 0 f117q.sanityn12 -> 0 f117q.sanityn13 -> 0 f117q.sanityn14 -> 1 f117q.sanityn15 -> 1 f117q.sanityn16 -> 0 f117q.sanityn17 -> 1 f117q.sanityn18 -> 0 f117q.sanityn19 -> 1 f117q.sanityn20 -> 1 f117q.sanityn21 -> 1 f117q.sanityn22 -> 1 f117q.sanityn23 -> 1 f117q.sanityn24 -> 1 f117q.sanityn25 -> 0 f117q.sanityn26 -> 1 f117q.sanityn27 -> 0 f117q.sanityn28 -> 1 f117q.sanityn29 -> 1 f117q.sanityn30 -> 1 f117q.sanityn31 -> 1 f117q.sanityn32 -> 0 f117q.sanityn33 -> 1 f117q.sanityn34 -> 0 f117q.sanityn35 -> 0 f117q.sanityn36 -> 1 f117q.sanityn37 -> 0 f117q.sanityn38 -> 1 f117q.sanityn39 -> 0 f117q.sanityn40 -> 1 f117q.sanityn41 -> 0 f117q.sanityn42 -> 1 f117q.sanityn43 -> 1 f117q.sanityn44 -> 0 f117q.sanityn45 -> 0 f117q.sanityn46 -> 0 f117q.sanityn47 -> 1 f117q.sanityn48 -> 1 f117q.sanityn49 -> 0 f117q.sanityn50 -> 0 f117q.sanityn51 -> 0 f117q.sanityn52 -> 1 f117q.sanityn53 -> 1 f117q.sanityn54 -> 1 f117q.sanityn55 -> 0 f117q.sanityn56 -> 1 f117q.sanityn57 -> 1 f117q.sanityn58 -> 1 f117q.sanityn59 -> 0 f117q.sanityn60 -> 1 f117q.sanityn61 -> 1 f117q.sanityn62 -> 1 f117q.sanityn63 -> 1 f117q.sanityn64 -> 1 f117q.sanityn65 -> 0 f117q.sanityn66 -> 1 f117q.sanityn67 -> 1 f117q.sanityn68 -> 0 f117q.sanityn69 -> 0 f117q.sanityn70 -> 0 f117q.sanityn71 -> 0 f117q.sanityn72 -> 0 f117q.sanityn73 -> 0 f117q.sanityn74 -> 0 f117q.sanityn75 -> 0 f117q.sanityn76 -> 1 f117q.sanityn77 -> 0 f117q.sanityn78 -> 1 f117q.sanityn79 -> 0 f117q.sanityn80 -> 1 f117q.sanityn81 -> 1 f117q.sanityn82 -> 0 f117q.sanityn83 -> 0 f117q.sanityn84 -> 1 f117q.sanityn85 -> 0 f117q.sanityn86 -> 1 f117q.sanityn87 -> 1 f117q.sanityn88 -> 1 f117q.sanityn89 -> 0 f117q.sanityn90 -> 1 f117q.sanityn91 -> 0 f117q.sanityn92 -> 0 f117q.sanityn93 -> 1 f117q.sanityn94 -> 0 f117q.sanityn95 -> 1 f117q.sanityn96 -> 0 f117q.sanityn97 -> 1 f117q.sanityn98 -> 0 f117q.sanityn99 -> 0 - unlinked 0 (time 1766396806 ; total 0 ; last 0) total: 100 unlinks in 0 seconds: inf unlinks/second f117q.sanityn0 f117q.sanityn1 f117q.sanityn10 f117q.sanityn11 f117q.sanityn12 f117q.sanityn13 f117q.sanityn14 f117q.sanityn15 f117q.sanityn16 f117q.sanityn17 f117q.sanityn18 f117q.sanityn19 f117q.sanityn2 f117q.sanityn20 f117q.sanityn21 f117q.sanityn22 f117q.sanityn23 f117q.sanityn24 f117q.sanityn25 f117q.sanityn26 f117q.sanityn27 f117q.sanityn28 f117q.sanityn29 f117q.sanityn3 f117q.sanityn30 f117q.sanityn31 f117q.sanityn32 f117q.sanityn33 f117q.sanityn34 f117q.sanityn35 f117q.sanityn36 f117q.sanityn37 f117q.sanityn38 f117q.sanityn39 f117q.sanityn4 f117q.sanityn40 f117q.sanityn41 f117q.sanityn42 f117q.sanityn43 f117q.sanityn44 f117q.sanityn45 f117q.sanityn46 f117q.sanityn47 f117q.sanityn48 f117q.sanityn49 f117q.sanityn5 f117q.sanityn50 f117q.sanityn51 f117q.sanityn52 f117q.sanityn53 f117q.sanityn54 f117q.sanityn55 f117q.sanityn56 f117q.sanityn57 f117q.sanityn58 f117q.sanityn59 f117q.sanityn6 f117q.sanityn60 f117q.sanityn61 f117q.sanityn62 f117q.sanityn63 f117q.sanityn64 f117q.sanityn65 f117q.sanityn66 f117q.sanityn67 f117q.sanityn68 f117q.sanityn69 f117q.sanityn7 f117q.sanityn70 f117q.sanityn71 f117q.sanityn72 f117q.sanityn73 f117q.sanityn74 f117q.sanityn75 f117q.sanityn76 f117q.sanityn77 f117q.sanityn78 f117q.sanityn79 f117q.sanityn8 f117q.sanityn80 f117q.sanityn81 f117q.sanityn82 f117q.sanityn83 f117q.sanityn84 f117q.sanityn85 f117q.sanityn86 f117q.sanityn87 f117q.sanityn88 f117q.sanityn89 f117q.sanityn9 f117q.sanityn90 f117q.sanityn91 f117q.sanityn92 f117q.sanityn93 f117q.sanityn94 f117q.sanityn95 f117q.sanityn96 f117q.sanityn97 f117q.sanityn98 f117q.sanityn99 f117q.sanityn10 f117q.sanityn11 f117q.sanityn12 f117q.sanityn13 f117q.sanityn14 f117q.sanityn15 f117q.sanityn16 f117q.sanityn17 f117q.sanityn18 f117q.sanityn19 f117q.sanityn20 f117q.sanityn21 f117q.sanityn22 f117q.sanityn23 f117q.sanityn24 f117q.sanityn25 f117q.sanityn26 f117q.sanityn27 f117q.sanityn28 f117q.sanityn29 f117q.sanityn30 f117q.sanityn31 f117q.sanityn32 f117q.sanityn33 f117q.sanityn34 f117q.sanityn35 f117q.sanityn36 f117q.sanityn37 f117q.sanityn38 f117q.sanityn39 f117q.sanityn40 f117q.sanityn41 f117q.sanityn42 f117q.sanityn43 f117q.sanityn44 f117q.sanityn45 f117q.sanityn46 f117q.sanityn47 f117q.sanityn48 f117q.sanityn49 f117q.sanityn50 f117q.sanityn51 f117q.sanityn52 f117q.sanityn53 f117q.sanityn54 f117q.sanityn55 f117q.sanityn56 f117q.sanityn57 f117q.sanityn58 f117q.sanityn59 f117q.sanityn60 f117q.sanityn61 f117q.sanityn62 f117q.sanityn63 f117q.sanityn64 f117q.sanityn65 f117q.sanityn66 f117q.sanityn67 f117q.sanityn68 f117q.sanityn69 f117q.sanityn70 f117q.sanityn71 f117q.sanityn72 f117q.sanityn73 f117q.sanityn74 f117q.sanityn75 f117q.sanityn76 f117q.sanityn77 f117q.sanityn78 f117q.sanityn79 f117q.sanityn80 f117q.sanityn81 f117q.sanityn82 f117q.sanityn83 f117q.sanityn84 f117q.sanityn85 f117q.sanityn86 f117q.sanityn87 f117q.sanityn88 f117q.sanityn89 f117q.sanityn90 f117q.sanityn91 f117q.sanityn92 f117q.sanityn93 f117q.sanityn94 f117q.sanityn95 f117q.sanityn96 f117q.sanityn97 f117q.sanityn98 f117q.sanityn99 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117q (14s) == sanityn test 117r: Verify the deleting a striped directory on Trash Can ========================================================== 04:46:54 (1766396814) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain #0 -> 0x200000bd0:0x7f1:0x0 #1 -> 0x240000404:0x761:0x0 Creating 50 files under striped directory: /mnt/lustre/d117r.sanityn/striped_dir total: 50 open/close in 0.35 seconds: 143.83 ops/second f117r.sanityn0 -> 0 f117r.sanityn1 -> 0 f117r.sanityn2 -> 0 f117r.sanityn3 -> 0 f117r.sanityn4 -> 0 f117r.sanityn5 -> 0 f117r.sanityn6 -> 0 f117r.sanityn7 -> 0 f117r.sanityn8 -> 0 f117r.sanityn9 -> 0 f117r.sanityn10 -> 1 f117r.sanityn11 -> 1 f117r.sanityn12 -> 1 f117r.sanityn13 -> 1 f117r.sanityn14 -> 1 f117r.sanityn15 -> 0 f117r.sanityn16 -> 1 f117r.sanityn17 -> 1 f117r.sanityn18 -> 0 f117r.sanityn19 -> 1 f117r.sanityn20 -> 0 f117r.sanityn21 -> 0 f117r.sanityn22 -> 0 f117r.sanityn23 -> 0 f117r.sanityn24 -> 0 f117r.sanityn25 -> 1 f117r.sanityn26 -> 0 f117r.sanityn27 -> 1 f117r.sanityn28 -> 1 f117r.sanityn29 -> 1 f117r.sanityn30 -> 1 f117r.sanityn31 -> 0 f117r.sanityn32 -> 0 f117r.sanityn33 -> 0 f117r.sanityn34 -> 1 f117r.sanityn35 -> 0 f117r.sanityn36 -> 0 f117r.sanityn37 -> 0 f117r.sanityn38 -> 0 f117r.sanityn39 -> 1 f117r.sanityn40 -> 0 f117r.sanityn41 -> 1 f117r.sanityn42 -> 0 f117r.sanityn43 -> 1 f117r.sanityn44 -> 0 f117r.sanityn45 -> 1 f117r.sanityn46 -> 0 f117r.sanityn47 -> 0 f117r.sanityn48 -> 1 f117r.sanityn49 -> 0 - unlinked 0 (time 1766396818 ; total 0 ; last 0) total: 50 unlinks in 1 seconds: 50.000000 unlinks/second Check files in Trash Can: f117r.sanityn0 f117r.sanityn1 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn2 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn3 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn4 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 f117r.sanityn5 f117r.sanityn6 f117r.sanityn7 f117r.sanityn8 f117r.sanityn9 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 FID for /mnt/lustre/d117r.sanityn/striped_dir [0x200000bd1:0xc72:0x0] pfid[0]=0x200000bd0:0x7f1:0x0 [0x200000400:0x2:0x0] [0x200000bd0:0x7f2:0x0] pfid[1]=0x240000404:0x761:0x0 [0x240000400:0x2:0x0] [0x240000400:0xe51:0x0] Obtain LMVEA for virtual dir /mnt/lustre/d117r.sanityn/striped_dir/.Trash: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x7f2:0x0] 1 [0x240000400:0xe51:0x0] #0 -> name: 0x200000bd0:0x7f1:0x0 fid: 0x200000bd0:0x7f2:0x0 #1 -> name: 0x240000404:0x761:0x0 fid: 0x240000400:0xe51:0x0 /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: striped_dir /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0/striped_dir: f117r.sanityn0 f117r.sanityn1 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn2 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn3 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn4 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 f117r.sanityn5 f117r.sanityn6 f117r.sanityn7 f117r.sanityn8 f117r.sanityn9 file.1 file.2 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: Obtain LMVEA for striped dir /mnt/lustre/d117r.sanityn/striped_dir: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x7f2:0x0] 1 [0x240000400:0xe51:0x0] [0x200000bd0:0x7f4:0x0] #0 -> name: 0x200000bd0:0x7f1:0x0 fid: 0x200000bd0:0x7f2:0x0 #1 -> name: 0x240000404:0x761:0x0 fid: 0x240000400:0xe51:0x0 lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x7f2:0x0] 1 [0x240000400:0xe51:0x0] f117r.sanityn0 f117r.sanityn1 f117r.sanityn10 f117r.sanityn11 f117r.sanityn12 f117r.sanityn13 f117r.sanityn14 f117r.sanityn15 f117r.sanityn16 f117r.sanityn17 f117r.sanityn18 f117r.sanityn19 f117r.sanityn2 f117r.sanityn20 f117r.sanityn21 f117r.sanityn22 f117r.sanityn23 f117r.sanityn24 f117r.sanityn25 f117r.sanityn26 f117r.sanityn27 f117r.sanityn28 f117r.sanityn29 f117r.sanityn3 f117r.sanityn30 f117r.sanityn31 f117r.sanityn32 f117r.sanityn33 f117r.sanityn34 f117r.sanityn35 f117r.sanityn36 f117r.sanityn37 f117r.sanityn38 f117r.sanityn39 f117r.sanityn4 f117r.sanityn40 f117r.sanityn41 f117r.sanityn42 f117r.sanityn43 f117r.sanityn44 f117r.sanityn45 f117r.sanityn46 f117r.sanityn47 f117r.sanityn48 f117r.sanityn49 f117r.sanityn5 f117r.sanityn6 f117r.sanityn7 f117r.sanityn8 f117r.sanityn9 file.1 file.2 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn0, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn1, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn10, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn11, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn12, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn13, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn14, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn15, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn16, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn17, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn18, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn19, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn2, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn20, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn21, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn22, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn23, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn24, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn25, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn26, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn27, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn28, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn29, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn3, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn30, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn31, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn32, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn33, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn34, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn35, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn36, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn37, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn38, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn39, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn4, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn40, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn41, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn42, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn43, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn44, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn45, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396818 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn46, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396818 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn47, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396818 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn48, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396818 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn49, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396818 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn5, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn6, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn7, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn8, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/f117r.sanityn9, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/file.1, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 file: /mnt/lustre/d117r.sanityn/striped_dir/file.2, flags: 0X80000, uid: 0 gid: 0 projid: 0 timestamp: 1766396817 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117r (14s) == sanityn test 117s: Create trash shard if it does not exist ========================================================== 04:47:08 (1766396828) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: file.1 /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x200000bd0:0x7f7:0x0] 1 [0x240000404:0x762:0x0] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117s (14s) == sanityn test 117t: UID/GID/ProjID for files in Trash Can ========================================================== 04:47:23 (1766396843) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain file: /mnt/lustre/d117t.sanityn/.Trash/tfile1, flags: 0X80002 unrm, uid: 0 gid: 0 projid: 0 timestamp: 1766396849 File: /mnt/lustre/d117t.sanityn/.Trash/tfile1 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826937494 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-12-22 04:47:29.000000000 -0500 Modify: 2025-12-22 04:47:29.000000000 -0500 Change: 2025-12-22 04:47:29.000000000 -0500 Birth: 2025-12-22 04:47:29.000000000 -0500 trash_can_uid=4294967294 trash_can_gid=4294967294 trash_can_projid=4294967294 === Checking ugp ID on MDT === oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519308 Type: regular Mode: 0644 Flags: 0x80002 Generation: 521560655 Version: 0x00000002:00018d3e User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x694913b1:00000000 -- Mon Dec 22 04:47:29 2025 atime: 0x694913b1:00000000 -- Mon Dec 22 04:47:29 2025 mtime: 0x694913b1:00000000 -- Mon Dec 22 04:47:29 2025 crtime: 0x694913b1:db956100 -- Mon Dec 22 04:47:29 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200000bd1:0xc96:0x0] compat=0 incompat=0 user.job (5) = "lfs.0" linkea: idx=0 parent=[0x200000bd0:0x7fb:0x0] name='tfile1' trusted.lov (56) = d0 0b d1 0b 01 00 00 00 96 0c 00 00 00 00 00 00 d1 0b 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 8f 19 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.unrm (24) = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b1 13 49 69 00 00 00 00 EXTENTS: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) === Checking ugp ID on OST === oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 3480 Type: regular Mode: 0666 Flags: 0x80000 Generation: 374957566 Version: 0x00000000:00000000 User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x00000000:00000000 -- Wed Dec 31 19:00:00 1969 atime: 0x00000000:00000000 -- Wed Dec 31 19:00:00 1969 mtime: 0x00000000:00000000 -- Wed Dec 31 19:00:00 1969 crtime: 0x69491384:ec08ce00 -- Mon Dec 22 04:46:44 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x280000401:0x198f:0x0] compat=8 incompat=0 EXTENTS: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117t.sanityn/tfile2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117t.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [unlink] [/mnt/lustre/d117t.sanityn/tfile2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [rmdir] [/mnt/lustre/d117t.sanityn/tdir2] file: /mnt/lustre/d117t.sanityn/.Trash/tfile2, flags: 0X80002 unrm, uid: 500 gid: 500 projid: 0 timestamp: 1766396862 File: /mnt/lustre/d117t.sanityn/.Trash/tfile2 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826937495 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 500/sanityusr) Gid: ( 500/sanityusr) Access: 2025-12-22 04:47:41.000000000 -0500 Modify: 2025-12-22 04:47:41.000000000 -0500 Change: 2025-12-22 04:47:41.000000000 -0500 Birth: 2025-12-22 04:47:41.000000000 -0500 using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre opts: Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg124-server tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg124-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg124-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg124-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/58765/testresults/sanityn-ldiskfs-DNE-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg124-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0000 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0001 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Started clients oleg124-client.virtnet: 192.168.201.124@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Started clients oleg124-client.virtnet: 192.168.201.124@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9f6fecaa9800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9f6ff7318800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6fecaa9800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6ff7318800.idle_timeout=debug disable quota as required mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117t (157s) == sanityn test 117u: statfs() call should not account the space in Trash Can ========================================================== 04:49:59 (1766396999) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_projid=1000 mdd.lustre-MDT0001.trash_can_projid=1000 llite.lustre-ffff9f6fecaa9800.trash_can_projid=1000 llite.lustre-ffff9f6ff7318800.trash_can_projid=1000 total=7666232 used=87332 avail=7126708 free=7578900 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.124@tcp:/lustre 7666232 87332 7126708 2% /mnt/lustre 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.64404 s, 63.8 MB/s new total=7666232 used=0 avail=7666232 free=7666232 Filesystem 1024-blocks Used Available Capacity Mounted on 192.168.201.124@tcp:/lustre 7666232 0 7666232 0% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 5624 1265680 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 4724 1266580 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 27860 3579160 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 161876 3445144 5% /mnt/lustre[OST:1] filesystem_summary: 7666232 189736 7024304 3% /mnt/lustre /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: f117u.sanityn /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: PROJID: 1000 Quota Info: Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 102404 0 0 - 1 0 0 - lustre-MDT0000 0 - 0 - 1 - 0 - lustre-MDT0001 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 102404 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for usr green (uid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-MDT0001 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 total=7666232 used=189736 avail=7024304 free=7476496 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 5624 1265680 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 4724 1266580 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 27860 3579160 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 161876 3445144 5% /mnt/lustre[OST:1] filesystem_summary: 7666232 189736 7024304 3% /mnt/lustre new total=7666232 used=27860 avail=7412276 free=7638372 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 5624 1265680 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 4724 1266580 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 27860 3579160 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 0 3833116 0% /mnt/lustre[OST:1] filesystem_summary: 7666232 27860 7412276 1% /mnt/lustre llite.lustre-ffff9f6fecaa9800.trash_can_projid=4294967294 llite.lustre-ffff9f6ff7318800.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_uid=4294967294 mdd.lustre-MDT0001.trash_can_uid=4294967294 mdd.lustre-MDT0000.trash_can_gid=4294967294 mdd.lustre-MDT0001.trash_can_gid=4294967294 mdd.lustre-MDT0000.trash_can_projid=4294967294 mdd.lustre-MDT0001.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117u (15s) == sanityn test 117ub: statfs() account the inode usage in Trash Can ========================================================== 04:50:14 (1766397014) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 12477 1011523 2% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1024000 9381 1014619 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 262144 3679 258465 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 262144 3681 258463 2% /mnt/lustre[OST:1] filesystem_summary: 538786 21858 516928 5% /mnt/lustre total: 1000 open/close in 5.47 seconds: 182.83 ops/second - unlinked 0 (time 1766397026 ; total 0 ; last 0) total: 1000 unlinks in 2 seconds: 500.000000 unlinks/second === w/o Trash Can account === lustre-MDT0000_UUID 1024000 13479 1010521 2% /mnt/lustre[MDT:0] === w/ Trash Can account === lustre-MDT0000_UUID 1024000 12479 1011521 2% /mnt/lustre[MDT:0] tcu_ffree: 1011521 ffree: 1010521 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117ub (28s) == sanityn test 117v: Test for Trash Can UID/GID/PROJID setting ========================================================== 04:50:42 (1766397042) mdd.lustre-MDT0000.trash_can_uid=10 mdd.lustre-MDT0001.trash_can_uid=10 mdd.lustre-MDT0000.trash_can_gid=10 mdd.lustre-MDT0001.trash_can_gid=10 mdd.lustre-MDT0000.trash_can_projid=10 mdd.lustre-MDT0001.trash_can_projid=10 llite.lustre-ffff9f6fecaa9800.trash_can_projid=10 llite.lustre-ffff9f6ff7318800.trash_can_projid=10 mdd.lustre-MDT0000.trash_can_uid=-1 mdd.lustre-MDT0001.trash_can_uid=-1 mdd.lustre-MDT0000.trash_can_gid=-1 mdd.lustre-MDT0001.trash_can_gid=-1 mdd.lustre-MDT0000.trash_can_projid=-1 mdd.lustre-MDT0001.trash_can_projid=-1 llite.lustre-ffff9f6fecaa9800.trash_can_projid=-1 llite.lustre-ffff9f6ff7318800.trash_can_projid=-1 mdd.lustre-MDT0000.trash_can_uid=-2 mdd.lustre-MDT0001.trash_can_uid=-2 mdd.lustre-MDT0000.trash_can_gid=-2 mdd.lustre-MDT0001.trash_can_gid=-2 mdd.lustre-MDT0000.trash_can_projid=-2 mdd.lustre-MDT0001.trash_can_projid=-2 llite.lustre-ffff9f6fecaa9800.trash_can_projid=-2 llite.lustre-ffff9f6ff7318800.trash_can_projid=-2 mdd.lustre-MDT0000.trash_can_uid=original mdd.lustre-MDT0001.trash_can_uid=original mdd.lustre-MDT0000.trash_can_gid=original mdd.lustre-MDT0001.trash_can_gid=original mdd.lustre-MDT0000.trash_can_projid=original mdd.lustre-MDT0001.trash_can_projid=original llite.lustre-ffff9f6fecaa9800.trash_can_projid=original llite.lustre-ffff9f6ff7318800.trash_can_projid=original mdd.lustre-MDT0000.trash_can_uid=default mdd.lustre-MDT0001.trash_can_uid=default mdd.lustre-MDT0000.trash_can_gid=default mdd.lustre-MDT0001.trash_can_gid=default mdd.lustre-MDT0000.trash_can_projid=default mdd.lustre-MDT0001.trash_can_projid=default llite.lustre-ffff9f6fecaa9800.trash_can_projid=default llite.lustre-ffff9f6ff7318800.trash_can_projid=default llite.lustre-ffff9f6fecaa9800.trash_can_projid=4294967294 llite.lustre-ffff9f6ff7318800.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_uid=4294967294 mdd.lustre-MDT0001.trash_can_uid=4294967294 mdd.lustre-MDT0000.trash_can_gid=4294967294 mdd.lustre-MDT0001.trash_can_gid=4294967294 mdd.lustre-MDT0000.trash_can_projid=4294967294 mdd.lustre-MDT0001.trash_can_projid=4294967294 PASS 117v (12s) == sanityn test 117w: Test Trash Can with original UID/GID/PROJID ========================================================== 04:50:54 (1766397054) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_uid=original mdd.lustre-MDT0001.trash_can_uid=original mdd.lustre-MDT0000.trash_can_gid=original mdd.lustre-MDT0001.trash_can_gid=original mdd.lustre-MDT0000.trash_can_projid=original mdd.lustre-MDT0001.trash_can_projid=original 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.40294 s, 74.7 MB/s /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x5:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: tdir.1 tfile.1 /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0/tdir.1: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519315 Type: regular Mode: 0644 Flags: 0x80002 Generation: 739157953 Version: 0x00000003:000010bb User: 0 Group: 0 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491484:00000000 -- Mon Dec 22 04:51:00 2025 atime: 0x69491482:00000000 -- Mon Dec 22 04:50:58 2025 mtime: 0x69491484:00000000 -- Mon Dec 22 04:51:00 2025 crtime: 0x69491483:4cc55200 -- Mon Dec 22 04:50:59 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3f0:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 f0 03 00 00 00 00 00 00 a1 13 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 97 1b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 user.job (4) = "dd.0" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tfile.1' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 40 06 00 00 00 00 00 20 03 00 00 00 00 00 EXTENTS: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519314 Type: directory Mode: 0755 Flags: 0x2 Generation: 1236992221 Version: 0x00000003:000010ba User: 0 Group: 0 Project: 0 Size: 4096 File ACL: 0 Links: 4 Blockcount: 8 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491484:00000000 -- Mon Dec 22 04:51:00 2025 atime: 0x69491482:00000000 -- Mon Dec 22 04:50:58 2025 mtime: 0x69491482:00000000 -- Mon Dec 22 04:50:58 2025 crtime: 0x69491483:4a62f800 -- Mon Dec 22 04:50:59 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3ef:0x0] compat=0 incompat=0 user.job (7) = "mkdir.0" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tdir.1' BLOCKS: (0):345717 TOTAL: 1 oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117w.sanityn/tdir.2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117w.sanityn/tfile.2] oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519318 Type: regular Mode: 0644 Flags: 0x80002 Generation: 2502911214 Version: 0x00000003:000010f3 User: 500 Group: 500 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491488:00000000 -- Mon Dec 22 04:51:04 2025 atime: 0x69491488:00000000 -- Mon Dec 22 04:51:04 2025 mtime: 0x69491488:00000000 -- Mon Dec 22 04:51:04 2025 crtime: 0x69491488:c2129e00 -- Mon Dec 22 04:51:04 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3f2:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 f2 03 00 00 00 00 00 00 a1 13 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 c0 02 00 00 00 97 1b 00 00 00 00 00 00 00 00 00 00 01 00 00 00 user.job (9) = "touch.500" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tfile.2' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EXTENTS: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 519317 Type: directory Mode: 0755 Flags: 0x2 Generation: 3085754940 Version: 0x00000003:000010f2 User: 500 Group: 500 Project: 0 Size: 4096 File ACL: 0 Links: 4 Blockcount: 8 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x69491488:00000000 -- Mon Dec 22 04:51:04 2025 atime: 0x69491488:00000000 -- Mon Dec 22 04:51:04 2025 mtime: 0x69491488:00000000 -- Mon Dec 22 04:51:04 2025 crtime: 0x69491488:be7f1700 -- Mon Dec 22 04:51:04 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x3f1:0x0] compat=0 incompat=0 user.job (9) = "mkdir.500" linkea: idx=0 parent=[0x2000013a0:0x3:0x0] name='tdir.2' BLOCKS: (0):345719 TOTAL: 1 oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) mdd.lustre-MDT0000.trash_can_uid=4294967294 mdd.lustre-MDT0001.trash_can_uid=4294967294 mdd.lustre-MDT0000.trash_can_gid=4294967294 mdd.lustre-MDT0001.trash_can_gid=4294967294 mdd.lustre-MDT0000.trash_can_projid=4294967294 mdd.lustre-MDT0001.trash_can_projid=4294967294 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117w (17s) == sanityn test 117x: Unrm Files from Trash Can should also restore UID/GID/PROJID ========================================================== 04:51:11 (1766397071) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tdir1 trusted.unrm=0sAAAAAAAAAAAAAAAAAAAAAJQUSWkAAAAA getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tfile1 trusted.unrm=0sAAAAAAAAAAAAAAAAAAAAAJQUSWkAAAAA /mnt/lustre/d117x.sanityn/tdir1: trusted.unrm: No such attribute /mnt/lustre/d117x.sanityn/tfile1: trusted.unrm: No such attribute running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117x.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117x.sanityn/tfile2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [rmdir] [/mnt/lustre/d117x.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [unlink] [/mnt/lustre/d117x.sanityn/tfile2] getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tdir2 trusted.unrm=0sAAAAAPQBAAD0AQAAAAAAAJQUSWkAAAAA getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117x.sanityn/.Trash/tfile2 trusted.unrm=0sAAAAAPQBAAD0AQAAAAAAAJQUSWkAAAAA /mnt/lustre/d117x.sanityn/tdir2: trusted.unrm: No such attribute /mnt/lustre/d117x.sanityn/tfile2: trusted.unrm: No such attribute mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117x (9s) == sanityn test 117ya: Test with symbol links with Trash Can ========================================================== 04:51:20 (1766397080) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain total 8 lrwxrwxrwx 1 root root 32 Dec 22 04:51 lnkd5 -> /mnt/lustre/d117ya.sanityn/tdir1 lrwxrwxrwx 1 root root 33 Dec 22 04:51 lnkf4 -> /mnt/lustre/d117ya.sanityn/tfile3 drwxr-xr-x 3 root root 4096 Dec 22 04:51 tdir1 drwxr-xr-x 2 root root 4096 Dec 22 04:51 tdir2 -rw-r--r-- 1 root root 0 Dec 22 04:51 tfile3 After unlink /mnt/lustre/d117ya.sanityn/lnkf4 and /mnt/lustre/d117ya.sanityn/lnkd5 /mnt/lustre/d117ya.sanityn/.Trash: total 4 lrwxrwxrwx 1 root root 32 Dec 22 04:51 lnkd5 -> /mnt/lustre/d117ya.sanityn/tdir1 lrwxrwxrwx 1 root root 33 Dec 22 04:51 lnkf4 -> /mnt/lustre/d117ya.sanityn/tfile3 d--------- 3 root root 4096 Dec 22 04:51 tdir1 -rw-r--r-- 1 root root 0 Dec 22 04:51 tfile3 /mnt/lustre/d117ya.sanityn/.Trash/tdir1: total 4 drwxr-xr-x 4 root root 4096 Dec 22 04:51 tdir13 -rw-r--r-- 1 root root 0 Dec 22 04:51 tfile11 -rw-r--r-- 1 root root 0 Dec 22 04:51 tfile12 /mnt/lustre/d117ya.sanityn/.Trash/tdir1/tdir13: total 0 file: /mnt/lustre/d117ya.sanityn/.Trash/lnkf4, flags: 0X2 unrm, uid: 0 gid: 0 projid: 0 timestamp: 1766397084 file: /mnt/lustre/d117ya.sanityn/.Trash/lnkd5, flags: 0X2 unrm, uid: 0 gid: 0 projid: 0 timestamp: 1766397084 Unrm stub file 'lnkf4' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/lnkf4' is in Trash Can Path '/mnt/lustre/d117ya.sanityn/.Trash/lnkf4' is a symlink Handle symlink /mnt/lustre/d117ya.sanityn/.Trash/lnkf4 Unrm stub file 'lnkd5' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/lnkd5' is in Trash Can Path '/mnt/lustre/d117ya.sanityn/.Trash/lnkd5' is a symlink Handle symlink /mnt/lustre/d117ya.sanityn/.Trash/lnkd5 Unrm stub file 'tdir1' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/tdir1' is in Trash Can Set flags 0X2/0 OK: rc=0 Handle dir 0:'tdir13' under directory '/mnt/lustre/d117ya.sanityn/tdir1' Handle special file 0:/mnt/lustre/d117ya.sanityn/tdir1/tdir13 Handle file 0:'tfile11' under directory '/mnt/lustre/d117ya.sanityn/tdir1' Handle special file 0:/mnt/lustre/d117ya.sanityn/tdir1/tfile11 Handle file 0:'tfile12' under directory '/mnt/lustre/d117ya.sanityn/tdir1' Handle special file 0:/mnt/lustre/d117ya.sanityn/tdir1/tfile12 Unrm stub file 'tfile3' under directory '/mnt/lustre/d117ya.sanityn' Check if '/mnt/lustre/d117ya.sanityn/.Trash/tfile3' is in Trash Can Set flags 0X80002/0X80000 OK: rc=0 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117ya (8s) == sanityn test 117yb: Compare the directories containing symlinks after unrm recover ========================================================== 04:51:28 (1766397088) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Only in /etc/pam.d: password-auth Only in /etc/postfix: master.cf Only in /etc: subgid- Only in /etc: subuid sanityn test_117yb: @@@@@@ FAIL: /etc and /mnt/lustre/d117yb.sanityn/etc diff Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:8873:test_117yb() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:8875:main() Dumping lctl log to /tmp/testlogs//sanityn.test_117yb.*.1766397160.log rsync: chown "/tmp/testlogs/.sanityn.test_117yb.debug_log.oleg124-server.1766397160.log.CRdiml" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_117yb.dmesg.oleg124-server.1766397160.log.KwcW1K" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid FAIL 117yb (75s) == sanityn test 117yc: Restoring UID/GID/PROJID for symlinks works ========================================================== 04:52:43 (1766397163) using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg124-client.virtnet /mnt/lustre (opts:) Stopping client oleg124-client.virtnet /mnt/lustre opts: Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg124-server tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg124-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg124-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg124-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/58765/testresults/sanityn-ldiskfs-DNE-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.24,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg124-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg124-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0000 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0001 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Starting client oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre Started clients oleg124-client.virtnet: 192.168.201.124@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Starting client oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre2 Started clients oleg124-client.virtnet: 192.168.201.124@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9f6fc54be800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9f6ff08b3000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6fc54be800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9f6ff08b3000.idle_timeout=debug disable quota as required mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117yc.sanityn/tdir1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117yc.sanityn/tfile1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [ln] [-s] [/mnt/lustre/d117yc.sanityn/tfile1] [/mnt/lustre/d117yc.sanityn/lnkf3] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [ln] [-s] [/mnt/lustre/d117yc.sanityn/tdir1] [/mnt/lustre/d117yc.sanityn/lnkd4] getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117yc.sanityn/.Trash/lnkf3 trusted.unrm=0sAAAAAPQBAAD0AQAAyAAAADwVSWkAAAAA getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d117yc.sanityn/.Trash/lnkd4 trusted.unrm=0sAAAAAPQBAAD0AQAAAAAAADwVSWkAAAAA Unrm stub file 'lnkf3' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/lnkf3' is in Trash Can Path '/mnt/lustre/d117yc.sanityn/.Trash/lnkf3' is a symlink Handle symlink /mnt/lustre/d117yc.sanityn/.Trash/lnkf3 Unrm stub file 'tdir1' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/tdir1' is in Trash Can Set flags 0X2/0 OK: rc=0 Unrm stub file 'lnkd4' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/lnkd4' is in Trash Can Path '/mnt/lustre/d117yc.sanityn/.Trash/lnkd4' is a symlink Handle symlink /mnt/lustre/d117yc.sanityn/.Trash/lnkd4 Unrm stub file 'tfile1' under directory '/mnt/lustre/d117yc.sanityn' Check if '/mnt/lustre/d117yc.sanityn/.Trash/tfile1' is in Trash Can Set flags 0X80002/0X80000 OK: rc=0 /mnt/lustre/d117yc.sanityn/lnkf3: trusted.unrm: No such attribute /mnt/lustre/d117yc.sanityn/lnkd4: trusted.unrm: No such attribute mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117yc (86s) == sanityn test 117za: Test Trash Can for UID Trash Can type with nodemap ========================================================== 04:54:09 (1766397249) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 500 /mnt/lustre/.lustre/trash/MDT0000/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200001b71:0x6:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: On MGS 192.168.201.124, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.201.124, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200001b71:0x6:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0/0x200001b71:0x6:0x0: tfile.1 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117za.sanityn/tfile.2] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/0/0x200001b71:0x6:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0 500 /mnt/lustre/.lustre/trash/MDT0000/tnm/0: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0/0x200001b71:0x6:0x0: tfile.1 /mnt/lustre/.lustre/trash/MDT0000/tnm/500: 0x200001b71:0x6:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/500/0x200001b71:0x6:0x0: tfile.2 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 520217 Type: regular Mode: 0644 Flags: 0x80002 Generation: 1653105775 Version: 0x00000004:000000fd User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x6949155e:00000000 -- Mon Dec 22 04:54:38 2025 atime: 0x6949155e:00000000 -- Mon Dec 22 04:54:38 2025 mtime: 0x6949155e:00000000 -- Mon Dec 22 04:54:38 2025 crtime: 0x6949155e:e8383e00 -- Mon Dec 22 04:54:38 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200001b71:0x9:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 09 00 00 00 00 00 00 00 71 1b 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 c0 02 00 00 00 a3 1c 00 00 00 00 00 00 00 00 00 00 01 00 00 00 user.job (9) = "touch.500" linkea: idx=0 parent=[0x200001b70:0x7:0x0] name='tfile.2' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.unrm (24) = 00 00 00 00 f4 01 00 00 f4 01 00 00 00 00 00 00 5e 15 49 69 00 00 00 00 EXTENTS: file: /mnt/lustre/.lustre/trash/MDT0000/tnm/500/0x200001b71:0x6:0x0/tfile.2, flags: 0X80002 unrm, uid: 500 gid: 500 projid: 0 timestamp: 1766397278 On MGS 192.168.201.124, active = nodemap.active=0 waiting 10 secs for sync mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117za (46s) == sanityn test 117zb: Test Trash Can for PLAIN Trash Can type with nodemap ========================================================== 04:54:55 (1766397295) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 0x200001b71:0xa:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xa:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: On MGS 192.168.201.124, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.201.124, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 0x200001b71:0xa:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xa:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0x200001b71:0xa:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0x200001b71:0xa:0x0: tfile.1 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117zb.sanityn/tfile.2] /mnt/lustre/.lustre/trash: MDT0000 MDT0001 /mnt/lustre/.lustre/trash/MDT0000: 0 0x200000bd0:0x7ef:0x0 0x200000bd1:0xbcc:0x0 0x200000bd1:0xbd2:0x0 0x200000bd1:0xbe3:0x0 0x200000bd1:0xbe7:0x0 0x200000bd1:0xbec:0x0 0x200000bd1:0xbf3:0x0 0x200000bd1:0xc06:0x0 0x200000bd1:0xc0c:0x0 0x200000bd1:0xc12:0x0 0x200000bd1:0xc1c:0x0 0x200000bd1:0xc1f:0x0 0x200000bd1:0xc25:0x0 0x200000bd1:0xc28:0x0 0x200000bd1:0xc2e:0x0 0x200000bd1:0xc33:0x0 0x200000bd1:0xc37:0x0 0x200000bd1:0xc39:0x0 0x200000bd1:0xc93:0x0 0x200000bd1:0xc95:0x0 0x2000013a1:0x3:0x0 0x2000013a1:0x3ee:0x0 0x2000013a1:0x3f3:0x0 0x2000013a1:0x3f8:0x0 0x2000013a1:0x401:0x0 0x2000013a1:0x5:0x0 0x200001b71:0x1:0x0 0x200001b71:0xa:0x0 500 tnm /mnt/lustre/.lustre/trash/MDT0000/0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd0:0x7ef:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbcc:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbd2:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbe7:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbec:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xbf3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc06:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc0c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc12:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1c:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc1f:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc25:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc28:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc2e:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc33:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc37:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc39:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc93:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200000bd1:0xc95:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3ee:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f3:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x3f8:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x401:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x2000013a1:0x5:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0x1:0x0: /mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xa:0x0: tfile.0 /mnt/lustre/.lustre/trash/MDT0000/500: /mnt/lustre/.lustre/trash/MDT0000/tnm: 0x200001b71:0xa:0x0 /mnt/lustre/.lustre/trash/MDT0000/tnm/0x200001b71:0xa:0x0: tfile.1 tfile.2 /mnt/lustre/.lustre/trash/MDT0001: 0x240000404:0x760:0x0 0x240000404:0x762:0x0 0x240000405:0x41:0x0 0x240000405:0xc:0x0 /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x760:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000404:0x762:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0x41:0x0: /mnt/lustre/.lustre/trash/MDT0001/0x240000405:0xc:0x0: oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 520217 Type: regular Mode: 0644 Flags: 0x80002 Generation: 921480410 Version: 0x00000004:0000021e User: -2 Group: -2 Project: -2 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x6949158c:00000000 -- Mon Dec 22 04:55:24 2025 atime: 0x6949158c:00000000 -- Mon Dec 22 04:55:24 2025 mtime: 0x6949158c:00000000 -- Mon Dec 22 04:55:24 2025 crtime: 0x6949158c:7a8c1200 -- Mon Dec 22 04:55:24 2025 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x200001b71:0xd:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 0d 00 00 00 00 00 00 00 71 1b 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 a5 1c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 user.job (9) = "touch.500" linkea: idx=0 parent=[0x200001b70:0x9:0x0] name='tfile.2' trusted.som (24) = 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 trusted.unrm (24) = 00 00 00 00 f4 01 00 00 f4 01 00 00 00 00 00 00 8c 15 49 69 00 00 00 00 EXTENTS: file: /mnt/lustre/.lustre/trash/MDT0000/tnm//0x200001b71:0xa:0x0/tfile.2, flags: 0X80002 unrm, uid: 500 gid: 500 projid: 0 timestamp: 1766397324 On MGS 192.168.201.124, active = nodemap.active=0 waiting 10 secs for sync mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117zb (45s) == sanityn test 117zc: TCU: Test striped dir for UID Trash Can type ========================================================== 04:55:40 (1766397340) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid Creating 100 files under striped directory: /mnt/lustre/d117zc.sanityn/striped_dir total: 100 open/close in 0.56 seconds: 179.42 ops/second f117zc.sanityn0 -> 1 f117zc.sanityn1 -> 1 f117zc.sanityn2 -> 1 f117zc.sanityn3 -> 1 f117zc.sanityn4 -> 1 f117zc.sanityn5 -> 1 f117zc.sanityn6 -> 1 f117zc.sanityn7 -> 1 f117zc.sanityn8 -> 1 f117zc.sanityn9 -> 1 f117zc.sanityn10 -> 0 f117zc.sanityn11 -> 0 f117zc.sanityn12 -> 0 f117zc.sanityn13 -> 1 f117zc.sanityn14 -> 0 f117zc.sanityn15 -> 1 f117zc.sanityn16 -> 1 f117zc.sanityn17 -> 0 f117zc.sanityn18 -> 1 f117zc.sanityn19 -> 1 f117zc.sanityn20 -> 0 f117zc.sanityn21 -> 1 f117zc.sanityn22 -> 0 f117zc.sanityn23 -> 0 f117zc.sanityn24 -> 0 f117zc.sanityn25 -> 1 f117zc.sanityn26 -> 0 f117zc.sanityn27 -> 0 f117zc.sanityn28 -> 1 f117zc.sanityn29 -> 1 f117zc.sanityn30 -> 1 f117zc.sanityn31 -> 0 f117zc.sanityn32 -> 0 f117zc.sanityn33 -> 0 f117zc.sanityn34 -> 0 f117zc.sanityn35 -> 1 f117zc.sanityn36 -> 0 f117zc.sanityn37 -> 1 f117zc.sanityn38 -> 1 f117zc.sanityn39 -> 1 f117zc.sanityn40 -> 1 f117zc.sanityn41 -> 1 f117zc.sanityn42 -> 0 f117zc.sanityn43 -> 1 f117zc.sanityn44 -> 0 f117zc.sanityn45 -> 0 f117zc.sanityn46 -> 0 f117zc.sanityn47 -> 0 f117zc.sanityn48 -> 0 f117zc.sanityn49 -> 1 f117zc.sanityn50 -> 0 f117zc.sanityn51 -> 0 f117zc.sanityn52 -> 0 f117zc.sanityn53 -> 1 f117zc.sanityn54 -> 0 f117zc.sanityn55 -> 1 f117zc.sanityn56 -> 0 f117zc.sanityn57 -> 1 f117zc.sanityn58 -> 1 f117zc.sanityn59 -> 0 f117zc.sanityn60 -> 1 f117zc.sanityn61 -> 0 f117zc.sanityn62 -> 0 f117zc.sanityn63 -> 0 f117zc.sanityn64 -> 1 f117zc.sanityn65 -> 0 f117zc.sanityn66 -> 0 f117zc.sanityn67 -> 1 f117zc.sanityn68 -> 1 f117zc.sanityn69 -> 1 f117zc.sanityn70 -> 0 f117zc.sanityn71 -> 0 f117zc.sanityn72 -> 0 f117zc.sanityn73 -> 0 f117zc.sanityn74 -> 1 f117zc.sanityn75 -> 0 f117zc.sanityn76 -> 1 f117zc.sanityn77 -> 0 f117zc.sanityn78 -> 0 f117zc.sanityn79 -> 1 f117zc.sanityn80 -> 0 f117zc.sanityn81 -> 1 f117zc.sanityn82 -> 1 f117zc.sanityn83 -> 1 f117zc.sanityn84 -> 0 f117zc.sanityn85 -> 1 f117zc.sanityn86 -> 1 f117zc.sanityn87 -> 1 f117zc.sanityn88 -> 0 f117zc.sanityn89 -> 1 f117zc.sanityn90 -> 0 f117zc.sanityn91 -> 1 f117zc.sanityn92 -> 1 f117zc.sanityn93 -> 0 f117zc.sanityn94 -> 0 f117zc.sanityn95 -> 0 f117zc.sanityn96 -> 1 f117zc.sanityn97 -> 0 f117zc.sanityn98 -> 0 f117zc.sanityn99 -> 1 #0 -> 0x200001b70:0xa:0x0 #1 -> 0x2400013a0:0x1:0x0 - unlinked 0 (time 1766397345 ; total 0 ; last 0) total: 100 unlinks in 0 seconds: inf unlinks/second f117zc.sanityn0 f117zc.sanityn1 f117zc.sanityn10 f117zc.sanityn11 f117zc.sanityn12 f117zc.sanityn13 f117zc.sanityn14 f117zc.sanityn15 f117zc.sanityn16 f117zc.sanityn17 f117zc.sanityn18 f117zc.sanityn19 f117zc.sanityn2 f117zc.sanityn20 f117zc.sanityn21 f117zc.sanityn22 f117zc.sanityn23 f117zc.sanityn24 f117zc.sanityn25 f117zc.sanityn26 f117zc.sanityn27 f117zc.sanityn28 f117zc.sanityn29 f117zc.sanityn3 f117zc.sanityn30 f117zc.sanityn31 f117zc.sanityn32 f117zc.sanityn33 f117zc.sanityn34 f117zc.sanityn35 f117zc.sanityn36 f117zc.sanityn37 f117zc.sanityn38 f117zc.sanityn39 f117zc.sanityn4 f117zc.sanityn40 f117zc.sanityn41 f117zc.sanityn42 f117zc.sanityn43 f117zc.sanityn44 f117zc.sanityn45 f117zc.sanityn46 f117zc.sanityn47 f117zc.sanityn48 f117zc.sanityn49 f117zc.sanityn5 f117zc.sanityn50 f117zc.sanityn51 f117zc.sanityn52 f117zc.sanityn53 f117zc.sanityn54 f117zc.sanityn55 f117zc.sanityn56 f117zc.sanityn57 f117zc.sanityn58 f117zc.sanityn59 f117zc.sanityn6 f117zc.sanityn60 f117zc.sanityn61 f117zc.sanityn62 f117zc.sanityn63 f117zc.sanityn64 f117zc.sanityn65 f117zc.sanityn66 f117zc.sanityn67 f117zc.sanityn68 f117zc.sanityn69 f117zc.sanityn7 f117zc.sanityn70 f117zc.sanityn71 f117zc.sanityn72 f117zc.sanityn73 f117zc.sanityn74 f117zc.sanityn75 f117zc.sanityn76 f117zc.sanityn77 f117zc.sanityn78 f117zc.sanityn79 f117zc.sanityn8 f117zc.sanityn80 f117zc.sanityn81 f117zc.sanityn82 f117zc.sanityn83 f117zc.sanityn84 f117zc.sanityn85 f117zc.sanityn86 f117zc.sanityn87 f117zc.sanityn88 f117zc.sanityn89 f117zc.sanityn9 f117zc.sanityn90 f117zc.sanityn91 f117zc.sanityn92 f117zc.sanityn93 f117zc.sanityn94 f117zc.sanityn95 f117zc.sanityn96 f117zc.sanityn97 f117zc.sanityn98 f117zc.sanityn99 f117zc.sanityn10 f117zc.sanityn11 f117zc.sanityn12 f117zc.sanityn13 f117zc.sanityn14 f117zc.sanityn15 f117zc.sanityn16 f117zc.sanityn17 f117zc.sanityn18 f117zc.sanityn19 f117zc.sanityn20 f117zc.sanityn21 f117zc.sanityn22 f117zc.sanityn23 f117zc.sanityn24 f117zc.sanityn25 f117zc.sanityn26 f117zc.sanityn27 f117zc.sanityn28 f117zc.sanityn29 f117zc.sanityn30 f117zc.sanityn31 f117zc.sanityn32 f117zc.sanityn33 f117zc.sanityn34 f117zc.sanityn35 f117zc.sanityn36 f117zc.sanityn37 f117zc.sanityn38 f117zc.sanityn39 f117zc.sanityn40 f117zc.sanityn41 f117zc.sanityn42 f117zc.sanityn43 f117zc.sanityn44 f117zc.sanityn45 f117zc.sanityn46 f117zc.sanityn47 f117zc.sanityn48 f117zc.sanityn49 f117zc.sanityn50 f117zc.sanityn51 f117zc.sanityn52 f117zc.sanityn53 f117zc.sanityn54 f117zc.sanityn55 f117zc.sanityn56 f117zc.sanityn57 f117zc.sanityn58 f117zc.sanityn59 f117zc.sanityn60 f117zc.sanityn61 f117zc.sanityn62 f117zc.sanityn63 f117zc.sanityn64 f117zc.sanityn65 f117zc.sanityn66 f117zc.sanityn67 f117zc.sanityn68 f117zc.sanityn69 f117zc.sanityn70 f117zc.sanityn71 f117zc.sanityn72 f117zc.sanityn73 f117zc.sanityn74 f117zc.sanityn75 f117zc.sanityn76 f117zc.sanityn77 f117zc.sanityn78 f117zc.sanityn79 f117zc.sanityn80 f117zc.sanityn81 f117zc.sanityn82 f117zc.sanityn83 f117zc.sanityn84 f117zc.sanityn85 f117zc.sanityn86 f117zc.sanityn87 f117zc.sanityn88 f117zc.sanityn89 f117zc.sanityn90 f117zc.sanityn91 f117zc.sanityn92 f117zc.sanityn93 f117zc.sanityn94 f117zc.sanityn95 f117zc.sanityn96 f117zc.sanityn97 f117zc.sanityn98 f117zc.sanityn99 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117zc (12s) == sanityn test 117zd: TCU: Test striped dir for UID Trash Can type with nodemap ========================================================== 04:55:52 (1766397352) On MGS 192.168.201.124, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.201.124, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid Creating 100 files under striped directory: /mnt/lustre/d117zd.sanityn/striped_dir total: 100 open/close in 0.56 seconds: 177.48 ops/second f117zd.sanityn0 -> 0 f117zd.sanityn1 -> 0 f117zd.sanityn2 -> 0 f117zd.sanityn3 -> 0 f117zd.sanityn4 -> 0 f117zd.sanityn5 -> 0 f117zd.sanityn6 -> 0 f117zd.sanityn7 -> 0 f117zd.sanityn8 -> 0 f117zd.sanityn9 -> 0 f117zd.sanityn10 -> 0 f117zd.sanityn11 -> 1 f117zd.sanityn12 -> 1 f117zd.sanityn13 -> 0 f117zd.sanityn14 -> 1 f117zd.sanityn15 -> 1 f117zd.sanityn16 -> 1 f117zd.sanityn17 -> 1 f117zd.sanityn18 -> 1 f117zd.sanityn19 -> 0 f117zd.sanityn20 -> 1 f117zd.sanityn21 -> 0 f117zd.sanityn22 -> 0 f117zd.sanityn23 -> 1 f117zd.sanityn24 -> 1 f117zd.sanityn25 -> 0 f117zd.sanityn26 -> 1 f117zd.sanityn27 -> 1 f117zd.sanityn28 -> 1 f117zd.sanityn29 -> 0 f117zd.sanityn30 -> 1 f117zd.sanityn31 -> 0 f117zd.sanityn32 -> 1 f117zd.sanityn33 -> 0 f117zd.sanityn34 -> 1 f117zd.sanityn35 -> 0 f117zd.sanityn36 -> 0 f117zd.sanityn37 -> 0 f117zd.sanityn38 -> 0 f117zd.sanityn39 -> 1 f117zd.sanityn40 -> 1 f117zd.sanityn41 -> 0 f117zd.sanityn42 -> 1 f117zd.sanityn43 -> 1 f117zd.sanityn44 -> 0 f117zd.sanityn45 -> 0 f117zd.sanityn46 -> 1 f117zd.sanityn47 -> 0 f117zd.sanityn48 -> 1 f117zd.sanityn49 -> 0 f117zd.sanityn50 -> 1 f117zd.sanityn51 -> 1 f117zd.sanityn52 -> 1 f117zd.sanityn53 -> 0 f117zd.sanityn54 -> 0 f117zd.sanityn55 -> 0 f117zd.sanityn56 -> 0 f117zd.sanityn57 -> 1 f117zd.sanityn58 -> 1 f117zd.sanityn59 -> 1 f117zd.sanityn60 -> 1 f117zd.sanityn61 -> 0 f117zd.sanityn62 -> 0 f117zd.sanityn63 -> 0 f117zd.sanityn64 -> 0 f117zd.sanityn65 -> 1 f117zd.sanityn66 -> 0 f117zd.sanityn67 -> 0 f117zd.sanityn68 -> 1 f117zd.sanityn69 -> 1 f117zd.sanityn70 -> 1 f117zd.sanityn71 -> 1 f117zd.sanityn72 -> 0 f117zd.sanityn73 -> 0 f117zd.sanityn74 -> 0 f117zd.sanityn75 -> 0 f117zd.sanityn76 -> 1 f117zd.sanityn77 -> 1 f117zd.sanityn78 -> 1 f117zd.sanityn79 -> 0 f117zd.sanityn80 -> 0 f117zd.sanityn81 -> 0 f117zd.sanityn82 -> 1 f117zd.sanityn83 -> 1 f117zd.sanityn84 -> 1 f117zd.sanityn85 -> 0 f117zd.sanityn86 -> 0 f117zd.sanityn87 -> 1 f117zd.sanityn88 -> 0 f117zd.sanityn89 -> 0 f117zd.sanityn90 -> 0 f117zd.sanityn91 -> 0 f117zd.sanityn92 -> 1 f117zd.sanityn93 -> 1 f117zd.sanityn94 -> 0 f117zd.sanityn95 -> 1 f117zd.sanityn96 -> 0 f117zd.sanityn97 -> 1 f117zd.sanityn98 -> 1 f117zd.sanityn99 -> 1 #0 -> 0x200001b70:0xe:0x0 #1 -> 0x2400013a0:0x4:0x0 - unlinked 0 (time 1766397382 ; total 0 ; last 0) total: 100 unlinks in 1 seconds: 100.000000 unlinks/second f117zd.sanityn0 f117zd.sanityn1 f117zd.sanityn10 f117zd.sanityn11 f117zd.sanityn12 f117zd.sanityn13 f117zd.sanityn14 f117zd.sanityn15 f117zd.sanityn16 f117zd.sanityn17 f117zd.sanityn18 f117zd.sanityn19 f117zd.sanityn2 f117zd.sanityn20 f117zd.sanityn21 f117zd.sanityn22 f117zd.sanityn23 f117zd.sanityn24 f117zd.sanityn25 f117zd.sanityn26 f117zd.sanityn27 f117zd.sanityn28 f117zd.sanityn29 f117zd.sanityn3 f117zd.sanityn30 f117zd.sanityn31 f117zd.sanityn32 f117zd.sanityn33 f117zd.sanityn34 f117zd.sanityn35 f117zd.sanityn36 f117zd.sanityn37 f117zd.sanityn38 f117zd.sanityn39 f117zd.sanityn4 f117zd.sanityn40 f117zd.sanityn41 f117zd.sanityn42 f117zd.sanityn43 f117zd.sanityn44 f117zd.sanityn45 f117zd.sanityn46 f117zd.sanityn47 f117zd.sanityn48 f117zd.sanityn49 f117zd.sanityn5 f117zd.sanityn50 f117zd.sanityn51 f117zd.sanityn52 f117zd.sanityn53 f117zd.sanityn54 f117zd.sanityn55 f117zd.sanityn56 f117zd.sanityn57 f117zd.sanityn58 f117zd.sanityn59 f117zd.sanityn6 f117zd.sanityn60 f117zd.sanityn61 f117zd.sanityn62 f117zd.sanityn63 f117zd.sanityn64 f117zd.sanityn65 f117zd.sanityn66 f117zd.sanityn67 f117zd.sanityn68 f117zd.sanityn69 f117zd.sanityn7 f117zd.sanityn70 f117zd.sanityn71 f117zd.sanityn72 f117zd.sanityn73 f117zd.sanityn74 f117zd.sanityn75 f117zd.sanityn76 f117zd.sanityn77 f117zd.sanityn78 f117zd.sanityn79 f117zd.sanityn8 f117zd.sanityn80 f117zd.sanityn81 f117zd.sanityn82 f117zd.sanityn83 f117zd.sanityn84 f117zd.sanityn85 f117zd.sanityn86 f117zd.sanityn87 f117zd.sanityn88 f117zd.sanityn89 f117zd.sanityn9 f117zd.sanityn90 f117zd.sanityn91 f117zd.sanityn92 f117zd.sanityn93 f117zd.sanityn94 f117zd.sanityn95 f117zd.sanityn96 f117zd.sanityn97 f117zd.sanityn98 f117zd.sanityn99 f117zd.sanityn10 f117zd.sanityn11 f117zd.sanityn12 f117zd.sanityn13 f117zd.sanityn14 f117zd.sanityn15 f117zd.sanityn16 f117zd.sanityn17 f117zd.sanityn18 f117zd.sanityn19 f117zd.sanityn20 f117zd.sanityn21 f117zd.sanityn22 f117zd.sanityn23 f117zd.sanityn24 f117zd.sanityn25 f117zd.sanityn26 f117zd.sanityn27 f117zd.sanityn28 f117zd.sanityn29 f117zd.sanityn30 f117zd.sanityn31 f117zd.sanityn32 f117zd.sanityn33 f117zd.sanityn34 f117zd.sanityn35 f117zd.sanityn36 f117zd.sanityn37 f117zd.sanityn38 f117zd.sanityn39 f117zd.sanityn40 f117zd.sanityn41 f117zd.sanityn42 f117zd.sanityn43 f117zd.sanityn44 f117zd.sanityn45 f117zd.sanityn46 f117zd.sanityn47 f117zd.sanityn48 f117zd.sanityn49 f117zd.sanityn50 f117zd.sanityn51 f117zd.sanityn52 f117zd.sanityn53 f117zd.sanityn54 f117zd.sanityn55 f117zd.sanityn56 f117zd.sanityn57 f117zd.sanityn58 f117zd.sanityn59 f117zd.sanityn60 f117zd.sanityn61 f117zd.sanityn62 f117zd.sanityn63 f117zd.sanityn64 f117zd.sanityn65 f117zd.sanityn66 f117zd.sanityn67 f117zd.sanityn68 f117zd.sanityn69 f117zd.sanityn70 f117zd.sanityn71 f117zd.sanityn72 f117zd.sanityn73 f117zd.sanityn74 f117zd.sanityn75 f117zd.sanityn76 f117zd.sanityn77 f117zd.sanityn78 f117zd.sanityn79 f117zd.sanityn80 f117zd.sanityn81 f117zd.sanityn82 f117zd.sanityn83 f117zd.sanityn84 f117zd.sanityn85 f117zd.sanityn86 f117zd.sanityn87 f117zd.sanityn88 f117zd.sanityn89 f117zd.sanityn90 f117zd.sanityn91 f117zd.sanityn92 f117zd.sanityn93 f117zd.sanityn94 f117zd.sanityn95 f117zd.sanityn96 f117zd.sanityn97 f117zd.sanityn98 f117zd.sanityn99 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid On MGS 192.168.201.124, active = nodemap.active=0 waiting 10 secs for sync PASS 117zd (50s) == sanityn test 118Aa: TCU: Orphan object handling for a regular file or an empty dir ========================================================== 04:56:42 (1766397402) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) 520220 (4096) 0x200001b71:0x78:0x0 Waiting for MDT destroys to complete oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) 0 (4096) 0x200001b71:0x78:0x0 sanityn test_118Aa: @@@@@@ FAIL: 0x200001b71:0x78:0x0 should be removed from PENDING Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9184:test_118Aa() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9206:main() Dumping lctl log to /tmp/testlogs//sanityn.test_118Aa.*.1766397418.log rsync: chown "/tmp/testlogs/.sanityn.test_118Aa.debug_log.oleg124-server.1766397418.log.7hmFsJ" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_118Aa.dmesg.oleg124-server.1766397418.log.JTffIx" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid FAIL 118Aa (19s) == sanityn test 118Ab: TCU: Orphan object handling for non-emtpy dir ========================================================== 04:57:01 (1766397421) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) 64004 (12) . 2 (4084) .. 520222 (4096) 0x200001b71:0x7a:0x0 Waiting for MDT destroys to complete oleg124-server: debugfs 1.47.3-wc2 (11-Nov-2025) 64004 (12) . 2 (4084) .. 0 (4096) 0x200001b71:0x7a:0x0 sanityn test_118Ab: @@@@@@ FAIL: 0x200001b71:0x7a:0x0 should be removed/cleanup from PENDING Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9250:test_118Ab() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9254:main() Dumping lctl log to /tmp/testlogs//sanityn.test_118Ab.*.1766397434.log rsync: chown "/tmp/testlogs/.sanityn.test_118Ab.debug_log.oleg124-server.1766397434.log.Sr305H" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_118Ab.dmesg.oleg124-server.1766397434.log.8IsWPp" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid FAIL 118Ab (15s) == sanityn test 118Ba: TCU: 'oldest' repeated deletion for a regular file or empty dir ========================================================== 04:57:17 (1766397437) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=oldest mdd.lustre-MDT0001.trash_can_delete_policy=oldest llite.lustre-ffff9f6fc54be800.trash_file_read=1 llite.lustre-ffff9f6ff08b3000.trash_file_read=1 llite.lustre-ffff9f6fc54be800.trash_file_read=0 llite.lustre-ffff9f6ff08b3000.trash_file_read=0 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118Ba (7s) == sanityn test 118Bb: TCU: 'timestamp' repeated deletion policy ========================================================== 04:57:23 (1766397443) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp f118Bb.sanityn f118Bb.sanityn.2025-12-22T09:57:27 f118Bb.sanityn.2025-12-22T09:57:27.553927496 f118Bb.sanityn.2025-12-22T09:57:27.574024207 f118Bb.sanityn.2025-12-22T09:57:27.593340780 f118Bb.sanityn.2025-12-22T09:57:27.611507065 f118Bb.sanityn.2025-12-22T09:57:27.630985268 f118Bb.sanityn.2025-12-22T09:57:27.651425810 f118Bb.sanityn.2025-12-22T09:57:27.672073186 f118Bb.sanityn.2025-12-22T09:57:27.692841366 f118Bb.sanityn.2025-12-22T09:57:27.714180621 f118Bb.sanityn.2025-12-22T09:57:27.734831339 f118Bb.sanityn.2025-12-22T09:57:27.760061256 f118Bb.sanityn.2025-12-22T09:57:27.780584331 f118Bb.sanityn.2025-12-22T09:57:27.802779099 f118Bb.sanityn.2025-12-22T09:57:27.825655187 d118Bb.sanityn d118Bb.sanityn.2025-12-22T09:57:28 d118Bb.sanityn.2025-12-22T09:57:28.301246240 d118Bb.sanityn.2025-12-22T09:57:28.315490886 d118Bb.sanityn.2025-12-22T09:57:28.330209424 d118Bb.sanityn.2025-12-22T09:57:28.344387887 d118Bb.sanityn.2025-12-22T09:57:28.361522357 d118Bb.sanityn.2025-12-22T09:57:28.377926310 d118Bb.sanityn.2025-12-22T09:57:28.392546318 d118Bb.sanityn.2025-12-22T09:57:28.407722732 d118Bb.sanityn.2025-12-22T09:57:28.422405456 d118Bb.sanityn.2025-12-22T09:57:28.438613049 d118Bb.sanityn.2025-12-22T09:57:28.453843210 d118Bb.sanityn.2025-12-22T09:57:28.468429720 d118Bb.sanityn.2025-12-22T09:57:28.483640700 d118Bb.sanityn.2025-12-22T09:57:28.502494979 d118Bb.sanityn.mulevel d118Bb.sanityn.mulevel.2025-12-22T09:57:29 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.288807116 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.358498387 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.422304689 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.487982444 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.552033196 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.623589771 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.691019031 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.759350494 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.827738713 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.906793266 d118Bb.sanityn.mulevel.2025-12-22T09:57:29.987530132 d118Bb.sanityn.mulevel.2025-12-22T09:57:30 d118Bb.sanityn.mulevel.2025-12-22T09:57:30.155481230 d118Bb.sanityn.mulevel.2025-12-22T09:57:30.245955404 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118Bb (11s) == sanityn test 118C: TCU: add support for rename() ====== 04:57:34 (1766397454) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp f118C.sanityn f118C.sanityn f118C.sanityn f118C.sanityn.2025-12-22T09:57:38 f118C.sanityn.2025-12-22T09:57:38.973865285 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118C (8s) == sanityn test 118d: TCU: repeated deletion of same fname for a empty dir in DNE env ========================================================== 04:57:42 (1766397462) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp d118d.sanityn.0 d118d.sanityn.0.2025-12-22T09:57:46 d118d.sanityn.0.2025-12-22T09:57:46.363927772 d118d.sanityn.0.2025-12-22T09:57:46.394881923 d118d.sanityn.0.2025-12-22T09:57:46.432878446 d118d.sanityn.0.2025-12-22T09:57:46.464560759 d118d.sanityn.0.2025-12-22T09:57:46.500793115 d118d.sanityn.0.2025-12-22T09:57:46.526488310 d118d.sanityn.0.2025-12-22T09:57:46.556951431 d118d.sanityn.0.2025-12-22T09:57:46.584310392 mdd.lustre-MDT0000.trash_can_delete_policy=timestamp mdd.lustre-MDT0001.trash_can_delete_policy=timestamp mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118d (7s) == sanityn test 118e: TCU: Repeated deletion of same name for a non-empty remote dir ========================================================== 04:57:49 (1766397469) SKIP: sanityn test_118e needs 3 MDTs at least SKIP 118e (2s) == sanityn test 118f: Avoid caching negative dentry cache for dir in Trash Can ========================================================== 04:57:51 (1766397471) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 9492: tree: command not found stat: cannot statx '/mnt/lustre/.lustre/trash/MDT0000/0x200001b71:0xf0:0x0/f118f.sanityn.1': No such file or directory === 2nd rm /mnt/lustre/d118f.sanityn/f118f.sanityn.1 ==== total 8 d--------- 2 root root 4096 Dec 22 04:57 . d--------- 32 root root 4096 Dec 22 02:31 .. -rw-r--r-- 1 root root 0 Dec 22 04:57 f118f.sanityn.1 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118f (6s) == sanityn test 118g: TCU: Tolerate ENOSPC error bypass Trash Can ========================================================== 04:57:57 (1766397477) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 13407 1010593 2% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1024000 9400 1014600 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 262144 3949 258195 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 262144 3950 258194 2% /mnt/lustre[OST:1] filesystem_summary: 539196 22807 516389 5% /mnt/lustre - create 6934 (time 1766397490.84 total 10.00 last 693.34) - create 10000 (time 1766397497.58 total 16.75 last 454.46) - create 14757 (time 1766397507.58 total 26.75 last 475.66) - create 19887 (time 1766397517.58 total 36.75 last 512.97) - create 24590 (time 1766397527.58 total 46.75 last 470.25) - create 29769 (time 1766397537.59 total 56.75 last 517.82) - create 34376 (time 1766397547.59 total 66.75 last 460.69) - create 38798 (time 1766397557.59 total 76.75 last 442.12) - create 40000 (time 1766397559.84 total 79.01 last 532.76) - create 44154 (time 1766397569.85 total 89.01 last 415.33) - create 47774 (time 1766397579.85 total 99.01 last 362.00) - create 50000 (time 1766397585.00 total 104.16 last 432.24) - create 54578 (time 1766397595.00 total 114.16 last 457.72) total: 56925 create in 120.00 seconds: 474.36 ops/second - unlinked 0 (time 1766397602 ; total 0 ; last 0) - unlinked 10000 (time 1766397643 ; total 41 ; last 41) - unlinked 20000 (time 1766397679 ; total 77 ; last 36) - unlinked 30000 (time 1766397709 ; total 107 ; last 30) - unlinked 40000 (time 1766397737 ; total 135 ; last 28) - unlinked 50000 (time 1766397765 ; total 163 ; last 28) unlink(/mnt/lustre/d118g.sanityn/tf56925) error: No such file or directory total: 56925 unlinks in 179 seconds: 318.016754 unlinks/second - create 8956 (time 1766397975.77 total 10.00 last 895.57) - create 18407 (time 1766397985.77 total 20.00 last 945.06) - create 28642 (time 1766397995.77 total 30.00 last 1023.43) - create 39111 (time 1766398005.77 total 40.00 last 1046.81) - create 49663 (time 1766398015.77 total 50.00 last 1055.12) - create 59623 (time 1766398025.77 total 60.00 last 995.94) - create 69191 (time 1766398035.77 total 70.00 last 956.76) - create 79056 (time 1766398045.77 total 80.00 last 986.45) - create 88692 (time 1766398055.77 total 90.01 last 963.52) - create 98930 (time 1766398065.77 total 100.01 last 1023.72) - create 108768 (time 1766398075.77 total 110.01 last 983.76) total: 117128 create in 120.00 seconds: 976.06 ops/second - unlinked 0 (time 1766398088 ; total 0 ; last 0) - unlinked 10000 (time 1766398110 ; total 22 ; last 22) - unlinked 20000 (time 1766398129 ; total 41 ; last 19) - unlinked 30000 (time 1766398148 ; total 60 ; last 19) - unlinked 40000 (time 1766398168 ; total 80 ; last 20) - unlinked 50000 (time 1766398188 ; total 100 ; last 20) - unlinked 60000 (time 1766398207 ; total 119 ; last 19) - unlinked 70000 (time 1766398227 ; total 139 ; last 20) - unlinked 80000 (time 1766398248 ; total 160 ; last 21) - unlinked 90000 (time 1766398270 ; total 182 ; last 22) - unlinked 100000 (time 1766398292 ; total 204 ; last 22) - unlinked 110000 (time 1766398313 ; total 225 ; last 21) unlink(/mnt/lustre/d118g.sanityn/tf117128) error: No such file or directory total: 117128 unlinks in 242 seconds: 484.000000 unlinks/second mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 118g (1196s) == sanityn test 118h: TCU: run mdtest with TCU enabled === 05:17:53 (1766398673) SKIP: sanityn test_118h mdtest not found SKIP 118h (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 05:17:54 (1766398674) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=3 striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d200.sanityn fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0868 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 52.6024 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 84.3462 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 116.087 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 148.829 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 180.604 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f6fc54be800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9f6fc54be800.ost_server_uuid in IDLE state after 0 sec PASS 200 (187s) cleanup: ====================================================== == sanityn test complete, duration 10091 sec ============= 05:21:02 (1766398862) sanityn: FAIL: test_117yb /etc and /mnt/lustre/d117yb.sanityn/etc diff sanityn: FAIL: test_118Aa 0x200001b71:0x78:0x0 should be removed from PENDING sanityn: FAIL: test_118Ab 0x200001b71:0x7a:0x0 should be removed/cleanup from PENDING === sanityn: start cleanup 05:21:02 (1766398862) === Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg124-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 05:22:11 (1766398931) ===