-----============= acceptance-small: sanityn ============----- Mon Jun 1 10:14:51 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg437-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a skipping tests SLOW=no: 33a === sanityn: start setup 10:15:11 (1780323311) === Starting client oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 Mount client oleg437-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 Started clients oleg437-client.virtnet: 192.168.204.137@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg437-client.virtnet: executing check_config_client /mnt/lustre oleg437-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg437-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9a4f88538800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9a4f8b96c800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9a4f88538800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9a4f8b96c800.idle_timeout=debug disable quota as required oleg437-server: oleg437-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanityn: finish setup 10:15:36 (1780323336) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245914 s, 42.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8207] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 10:15:37 (1780323337) PASS 0 (5s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 10:15:42 (1780323342) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (4s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 10:15:46 (1780323346) -rw-r--r-- 1 root root 0 Jun 1 10:15 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 10:15:50 (1780323350) -rw-r--r-- 1 root root 0 Jun 1 10:15 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 10:15:54 (1780323354) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (5s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 10:15:59 (1780323359) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 10:16:03 (1780323363) total 0 -rwxrwxrwx 1 root root 0 Jun 1 10:15 f2a -rwxrwxrwx 1 root root 0 Jun 1 10:15 f2b total 0 -rwxrwxrwx 1 root root 0 Jun 1 10:15 f2a -rwxrwxrwx 1 root root 0 Jun 1 10:15 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0107093 s, 47.8 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 10:16:07 (1780323367) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (4s) == sanityn test 2g: check blocks update on sync write ==== 10:16:11 (1780323371) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.108496 s, 19.3 MB/s /mnt/lustre/f2g.sanityn has 4096 blocks /mnt/lustre2/f2g.sanityn has 4096 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 10:16:15 (1780323375) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 10:16:18 (1780323378) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 10:16:22 (1780323382) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 10:16:25 (1780323385) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 10:16:28 (1780323388) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 10:16:31 (1780323391) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 10:16:34 (1780323394) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 10:16:37 (1780323397) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240248 s, 43.6 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0267052 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 10:16:40 (1780323400) 1+0 records in 1+0 records out 1 byte copied, 0.0037775 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00517192 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00613996 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00439977 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00515615 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00564865 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00603265 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00617236 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00871237 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00549385 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00621966 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00531458 s, 0.2 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 10:16:43 (1780323403) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00568973 s, 540 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0202966 s, 202 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000245514 s, 12.5 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 10:16:46 (1780323406) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (3s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 10:16:49 (1780323409) warning: '-runtime' deprecated, use '-t runtime' instead - link 1599 (time 1780323420.70 total 10.00 last 159.83) - link 3154 (time 1780323430.70 total 20.01 last 155.45) - link 4375 (time 1780323440.71 total 30.02 last 121.99) - link 5477 (time 1780323450.71 total 40.02 last 110.19) - link 6651 (time 1780323460.71 total 50.02 last 117.37) - link 8365 (time 1780323470.71 total 60.02 last 171.37) - link 9615 (time 1780323480.72 total 70.02 last 124.98) - link 10000 (time 1780323483.48 total 72.78 last 139.46) - link 11284 (time 1780323493.49 total 82.79 last 128.28) - link 12461 (time 1780323503.49 total 92.80 last 117.68) total: 13478 link in 100.00 seconds: 134.78 ops/second using seed 1147568988 running for 100 seconds - stat 10000 (time 1780323417 ; total 6 ; last 6) - stat 20000 (time 1780323423 ; total 12 ; last 6) - stat 30000 (time 1780323428 ; total 17 ; last 5) - stat 40000 (time 1780323434 ; total 23 ; last 6) - stat 50000 (time 1780323440 ; total 29 ; last 6) - stat 60000 (time 1780323446 ; total 35 ; last 6) - stat 70000 (time 1780323452 ; total 41 ; last 6) - stat 80000 (time 1780323458 ; total 47 ; last 6) - stat 89778 (time 1780323469 ; total 58 ; last 11) - stat 90000 (time 1780323469 ; total 58 ; last 0) - stat 96426 (time 1780323480 ; total 69 ; last 11) - stat 100000 (time 1780323485 ; total 74 ; last 5) - stat 106147 (time 1780323496 ; total 85 ; last 11) - stat 110000 (time 1780323502 ; total 91 ; last 6) total: 115620 stats in 100 seconds: 1156.199951 stats/second - unlinked 0 (time 1780323461 ; total 0 ; last 0) - unlinked 10000 (time 1780323531 ; total 70 ; last 70) unlink(/mnt/lustre2/lockdir/lockfile13478) error: No such file or directory total: 13477 unlinks in 85 seconds: 158.552948 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22665) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22667) - No such process PASS 12 (139s) == sanityn test 13: test directory page revocation ======= 10:19:08 (1780323548) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d13 f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 10:19:10 (1780323550) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 23873 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 10:19:12 (1780323552) striped dir -i0 -c2 -H crush2 /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 24463 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 10:19:15 (1780323555) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 25053 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 10:19:17 (1780323557) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 25646 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 10:19:20 (1780323560) striped dir -i0 -c2 -H crush /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 26241 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 10:19:23 (1780323563) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky Waiting for MDT destroys to complete 0 STRIPECOUNT=2 ORIGFREE=7207284 MAXFREE=2097152000 0 lustre-OST0000 avl=2012 grnt=2695 diff=-683 limit=46331 FULL lustre-OST0001 avl=384 grnt=579 diff=-195 limit=46331 FULL ERROR: dd blocks 7197334 != file size 7197356 LOG LOG2 file /tmp/oos2.log:dd: error writing '/mnt/lustre/oosfile': No space left on device /tmp/oos2.log:3597710+0 records in /tmp/oos2.log:3597709+0 records out /tmp/oos2.log:3684054016 bytes (3.7 GB, 3.4 GiB) copied, 546.07 s, 6.7 MB/s /tmp/oos2.log2:dd: error writing '/mnt/lustre2/oosfile2': No space left on device /tmp/oos2.log2:3599626+0 records in /tmp/oos2.log2:3599625+0 records out /tmp/oos2.log2:3686016000 bytes (3.7 GB, 3.4 GiB) copied, 545.662 s, 6.8 MB/s Waiting for MDT destroys to complete Waiting for MDT destroys to complete PASS 15 (585s) == sanityn test 16a: 2500 iterations of dual-mount fsx === 10:29:08 (1780324148) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.15382 s, 54.5 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.153119 s, 54.8 MB/s Chance of close/open is 1 in 50 Seed set to 4151 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn truncating to largest ever: 0x2d92d4 truncating to largest ever: 0x5b3b59 truncating to largest ever: 0x6d7143 fallocating to largest ever: 0x7b3add fallocating to largest ever: 0x93f7c3 000100[0] 1780324155.963969 write 0x4e4372 thru 0x4e6b8e (0x281d bytes) truncating to largest ever: 0x99a643 000200[1] 1780324159.855001 trunc from 0x54bf02 to 0x55afe6 (0xf0e5 bytes) 000400[1] 1780324166.915394 mapread 0x1f9511 thru 0x2039f1 (0xa4e1 bytes) truncating to largest ever: 0x9bc3dd 000600[0] 1780324174.573925 punch from 0x1fd13e to 0x202840 (0x5703 bytes) 000700[1] 1780324177.516701 read 0x11fda6 thru 0x126adb (0x6d36 bytes) fallocating to largest ever: 0x9c4243 000800[0] 1780324180.843964 punch from 0x1437bd to 0x14be56 (0x869a bytes) fallocating to largest ever: 0x9e7491 001000[1] 1780324188.100741 trunc from 0x7665cc to 0x506459 (0x260172 bytes) 001100[0] 1780324191.600783 punch from 0x46b175 to 0x4792ea (0xe176 bytes) 001200[1] 1780324195.234996 trunc from 0x26b159 to 0x26f49d (0x4345 bytes) 001300[1] 1780324199.061846 punch from 0x54d894 to 0x5584c2 (0xac2f bytes) 001400[1] 1780324202.591813 trunc from 0x729aa6 to 0x98d6e4 (0x263c3f bytes) 001600[0] 1780324209.950116 write 0x322e77 thru 0x32f879 (0xca03 bytes) 001700[1] 1780324214.040048 punch from 0x2ad4f1 to 0x2babaf (0xd6bf bytes) 001800[0] 1780324217.113739 write 0x602eaa thru 0x60ea6c (0xbbc3 bytes) 001900[0] 1780324220.510277 read 0x6f785b thru 0x7017ce (0x9f74 bytes) 002000[0] 1780324223.291493 trunc from 0x4c9860 to 0x75113b (0x2878dc bytes) 002100[1] 1780324226.866891 mapread 0x6210b5 thru 0x62a191 (0x90dd bytes) fallocating to largest ever: 0x9e78b4 002300[1] 1780324234.178228 trunc from 0x3064ec to 0x7b7e0d (0x4b1922 bytes) fallocating to largest ever: 0x9e78e4 truncating to largest ever: 0x9edde4 002500[1] 1780324241.041710 read 0x0f24e2 thru 0x0ff023 (0xcb42 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 4241 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn truncating to largest ever: 0x5a0335 truncating to largest ever: 0x9967ff 000200[0] 1780324246.431105 write_OD 0x7a6000 thru 0x7aafff (0x5000 bytes) 000300[0] 1780324249.158071 fallocate 0x3d30a5 thru 0x3e11bf (0xe11b bytes) truncating to largest ever: 0x9c6a2b 000500[0] 1780324254.504144 trunc from 0x16fd96 to 0x17975d (0x99c8 bytes) truncating to largest ever: 0x9f703f 000600[1] 1780324257.241760 mapwrite 0x12b000 thru 0x12f193 (0x4194 bytes) 000700[1] 1780324260.268051 trunc from 0x837227 to 0x785727 (0xb1aff bytes) 000900[1] 1780324265.729329 fallocate 0x274ec6 thru 0x27acdd (0x5e18 bytes) 001000[0] 1780324268.653702 trunc from 0x377c78 to 0x37d927 (0x5cb0 bytes) 001100[0] 1780324272.192370 read_OD 0x211000 thru 0x21dfff (0xd000 bytes) fallocating to largest ever: 0x9f8326 001400[1] 1780324282.165186 mapread 0x3aa000 thru 0x3ab0ce (0x10cf bytes) 001500[1] 1780324285.451675 mapread 0x01c000 thru 0x02b12d (0xf12e bytes) 001700[1] 1780324291.790628 mapread 0x12e000 thru 0x130253 (0x2254 bytes) fallocating to largest ever: 0xa00000 001800[0] 1780324295.387019 trunc from 0x7bbf93 to 0x500551 (0x2bba41 bytes) 001900[1] 1780324298.678819 trunc from 0x27263b to 0x7a15b2 (0x52ef78 bytes) 002000[1] 1780324301.620826 trunc from 0x635f41 to 0x63dfab (0x806b bytes) 002100[1] 1780324304.681193 punch from 0x4aba56 to 0x4b8fb8 (0xd563 bytes) 002200[0] 1780324307.063057 trunc from 0x4a4ba1 to 0x930e7d (0x48c2dd bytes) 002300[0] 1780324309.301341 mapwrite 0x3f3000 thru 0x3f538d (0x238e bytes) 002400[1] 1780324311.582999 write 0x645000 thru 0x645cde (0xcdf bytes) 002500[0] 1780324314.191549 fallocate 0x21ed68 thru 0x22ea34 (0xfccd bytes) All operations completed A-OK! PASS 16a (169s) == sanityn test 16b: 2500 iterations of dual-mount fsx at small size ========================================================== 10:31:57 (1780324317) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.110959 s, 75.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0982996 s, 85.3 MB/s Chance of close/open is 1 in 50 Seed set to 4319 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fallocating to largest ever: 0x604bbf fallocating to largest ever: 0x7def3f truncating to largest ever: 0x7fc212 fallocating to largest ever: 0x9cf913 000100[1] 1780324322.448618 mapread 0x6f2332 thru 0x6f26e4 (0x3b3 bytes) 000200[1] 1780324325.354753 fallocate 0x3837ce thru 0x38564d (0x1e80 bytes) truncating to largest ever: 0x9e0298 truncating to largest ever: 0x9ee7be 000300[1] 1780324328.647751 mapwrite 0x55dd64 thru 0x55e506 (0x7a3 bytes) 000400[0] 1780324331.136817 punch from 0x6eef43 to 0x6efda4 (0xe62 bytes) 000500[1] 1780324334.189118 trunc from 0x8b42de to 0x1d3091 (0x6e124c bytes) 000600[0] 1780324336.841810 mapread 0x37c44d thru 0x37c9b7 (0x56b bytes) 000800[1] 1780324342.128558 read 0x54d181 thru 0x54d6cc (0x54c bytes) 000900[0] 1780324344.960506 trunc from 0x643a00 to 0x644a5d (0x105e bytes) 001100[0] 1780324350.130211 write 0x7d3956 thru 0x7d5686 (0x1d31 bytes) 001200[1] 1780324352.750229 read 0x2feef2 thru 0x3004e4 (0x15f3 bytes) 001500[1] 1780324360.645393 trunc from 0x9a363a to 0x27aa9d (0x728b9c bytes) 001600[1] 1780324363.610942 trunc from 0x1f5e9c to 0x37d8b4 (0x187a19 bytes) 001700[0] 1780324365.950921 fallocate 0x0a4df4 thru 0x0a5c30 (0xe3d bytes) 001800[0] 1780324369.007259 trunc from 0x971e70 to 0x99b8b6 (0x29a47 bytes) 001900[0] 1780324371.925401 trunc from 0x5b6a0f to 0x374294 (0x24277a bytes) 002000[1] 1780324374.787654 trunc from 0x112503 to 0x11412e (0x1c2c bytes) 002100[0] 1780324377.601131 mapwrite 0x4a0fc0 thru 0x4a198b (0x9cc bytes) truncating to largest ever: 0x9f9d47 002300[1] 1780324384.283707 trunc from 0x49e4af to 0x49fe8b (0x19dd bytes) fallocating to largest ever: 0xa00000 002500[0] 1780324391.362557 punch from 0x104b96 to 0x105bac (0x1017 bytes) All operations completed A-OK! PASS 16b (77s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 10:33:14 (1780324394) keep default fallocate mode: 0 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.122649 s, 68.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0972743 s, 86.2 MB/s Waiting for MDT destroys to complete Chance of close/open is 1 in 50 Seed set to 4408 fd 0: /mnt/lustre/f16c.sanityn fd 1: /mnt/lustre2/f16c.sanityn fallocating to largest ever: 0x5c1396 fallocating to largest ever: 0x799caa fallocating to largest ever: 0x97eb20 000100[1] 1780324412.072980 write 0x8fe429 thru 0x904a56 (0x662e bytes) 000200[0] 1780324415.246031 trunc from 0x91fc34 to 0x52bde0 (0x3f3e53 bytes) truncating to largest ever: 0x9c28d2 000300[0] 1780324418.758632 fallocate 0x3ffa6d thru 0x406568 (0x6afc bytes) fallocating to largest ever: 0x9dc229 zero_range to largest ever: 0x9e927c fallocating to largest ever: 0x9ffa0c 000500[1] 1780324425.103096 read 0x261705 thru 0x26b7c3 (0xa0bf bytes) 000600[0] 1780324428.089258 trunc from 0x3b688f to 0x3bfcf4 (0x9466 bytes) 000700[0] 1780324430.788121 trunc from 0x5f5c58 to 0x9701e9 (0x37a592 bytes) 000800[1] 1780324433.995570 fallocate 0x8c422c thru 0x8c60ed (0x1ec2 bytes) 000900[1] 1780324436.780982 write 0x38edda thru 0x39e96b (0xfb92 bytes) 001000[0] 1780324439.883635 mapread 0x6cdd46 thru 0x6db850 (0xdb0b bytes) 001200[1] 1780324446.487716 read 0x2f2ec7 thru 0x2f83ea (0x5524 bytes) 001300[1] 1780324449.067722 mapwrite 0x0afb1d thru 0x0bad43 (0xb227 bytes) 001400[1] 1780324451.618592 write 0x08c114 thru 0x095cdd (0x9bca bytes) 001500[0] 1780324454.622114 punch from 0x015a07 to 0x019d32 (0x432c bytes) 001600[1] 1780324458.197500 read 0x0544bd thru 0x0579bb (0x34ff bytes) 001700[0] 1780324461.039958 read 0x2c201e thru 0x2cdc1f (0xbc02 bytes) 001800[1] 1780324464.319140 read 0x36a3d4 thru 0x36d5f4 (0x3221 bytes) 001900[0] 1780324467.396841 mapwrite 0x34f57a thru 0x351291 (0x1d18 bytes) 002000[0] 1780324470.574285 trunc from 0x663143 to 0x61c54d (0x46bf5 bytes) 002100[1] 1780324473.700425 read 0x088296 thru 0x09272b (0xa496 bytes) 002300[1] 1780324480.936529 trunc from 0x7f140b to 0x7f7be8 (0x67de bytes) 002400[1] 1780324484.557333 write 0x4f0f8d thru 0x4fceac (0xbf20 bytes) 002500[1] 1780324488.282694 write 0x758d96 thru 0x75f0e7 (0x6352 bytes) All operations completed A-OK! PASS 16c (98s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 10:34:52 (1780324492) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.154858 s, 54.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.13512 s, 62.1 MB/s ldlm.namespaces.MGC192.168.204.137@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9a4f88538800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9a4f8b96c800.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9a4f88538800.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9a4f8b96c800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9a4f8b96c800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9a4f8b96c800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.05765 s, 25.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.87185 s, 56.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.64361 s, 63.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.49702 s, 30.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.46183 s, 42.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.963 s, 53.4 MB/s PASS 16d (23s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 10:35:15 (1780324515) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.165567 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287118 s, 36.5 MB/s PASS 16e (4s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 10:35:19 (1780324519) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 10:35:43 (1780324543) TIMEOUT PASS 16g (24s) == sanityn test 16h: mmap read after truncate file ======= 10:36:07 (1780324567) striped dir -i0 -c2 -H crush2 /mnt/lustre/d16h.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (4s) == sanityn test 16i: read after truncate file ============ 10:36:12 (1780324572) striped dir -i0 -c2 -H crush /mnt/lustre/d16i.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (4s) == sanityn test 16j: race dio with buffered i/o ========== 10:36:16 (1780324576) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.481048 s, 54.5 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.71905 s, 36.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.901193 s, 29.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.484389 s, 54.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.663188 s, 39.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.526974 s, 49.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.829353 s, 31.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.769438 s, 34.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.799352 s, 32.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.856164 s, 30.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.02708 s, 25.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.455999 s, 57.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.499428 s, 52.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.6313 s, 41.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.664424 s, 39.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.525411 s, 49.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.560094 s, 46.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.830526 s, 31.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.862712 s, 30.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (15s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 10:36:31 (1780324591) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fallocating to largest ever: 0x3428364 skipping zero size read fallocating to largest ever: 0x3428364 skipping zero size read fallocating to largest ever: 0x3428364 skipping zero size read fallocating to largest ever: 0x3428364 fallocating to largest ever: 0x3428364 skipping zero size read skipping zero size read truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3ddbbf6 fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee fallocating to largest ever: 0x3fa9cee truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 truncating to largest ever: 0x3fb80f5 All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Mon Jun 1 10:36:49 EDT 2026 + fsx 2 done: rc=0 Mon Jun 1 10:36:49 EDT 2026 + fsx 3 done: rc=0 Mon Jun 1 10:36:49 EDT 2026 + fsx 4 done: rc=0 Mon Jun 1 10:36:49 EDT 2026 + fsx 5 done: rc=0 Mon Jun 1 10:36:49 EDT 2026 PASS 16k (23s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 10:36:54 (1780324614) fail_loc=0x8000030a PASS 17 (6s) == sanityn test 18: mmap sanity check =========================================================================================== 10:37:00 (1780324620) mmap test1: basic mmap operation (PASS, 0.071785s) mmap test2: MAP_PRIVATE not write back (PASS, 0.062728s) mmap test3: concurrent mmap ops on two nodes (PASS, 9.3436s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.2743s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.10726s) mmap test6: check mmap write/read content on two nodes (PASS, 0.14774s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.70165s) mmap test11: multi-thread shared mmap (PASS, 0.72006s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0489s) PASS 18 (22s) == sanityn test 19: test concurrent uncached read races ========================================================================= 10:37:22 (1780324642) 32+0 records in 32+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.219692 s, 76.4 MB/s loop 5 loop 10 loop 15 loop 20 PASS 19 (17s) == sanityn test 20: test extra readahead page left in cache ============================================================== 10:37:39 (1780324659) striped dir -i0 -c2 -H all_char /mnt/lustre/d20.sanityn PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 10:37:42 (1780324662) striped dir -i1 -c2 -H crush /mnt/lustre/d21.sanityn rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 10:37:46 (1780324666) now is 1780324667 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 new atime is 1780324728 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 10:38:51 (1780324731) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2028 1269276 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 31704 3575316 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 8044 3598976 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 39748 7174292 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2028 1269276 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 31704 3575316 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 8044 3598976 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 39748 7174292 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 356 999.7K 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1000.0K 277 999.7K 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 256.0K 404 255.6K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 256.0K 403 255.6K 1% /mnt/lustre[OST:1] filesystem_summary: 511.8K 633 511.2K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 356 999.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1000.0K 277 999.7K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 256.0K 404 255.6K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 256.0K 403 255.6K 1% /mnt/lustre2[OST:1] filesystem_summary: 511.8K 633 511.2K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 1.3G 2.0M 1.2G 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1.3G 1.7M 1.2G 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3.7G 31.0M 3.4G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.7G 7.9M 3.4G 1% /mnt/lustre[OST:1] filesystem_summary: 7.3G 38.8M 6.8G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 356 1023644 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1024000 277 1023723 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 262144 404 261740 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 262144 403 261741 1% /mnt/lustre2[OST:1] filesystem_summary: 524114 633 523481 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2028 1269276 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 31704 3575316 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 8044 3598976 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 39748 7174292 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1000.0K 356 999.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1000.0K 277 999.7K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 256.0K 404 255.6K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 256.0K 403 255.6K 1% /mnt/lustre2[OST:1] filesystem_summary: 511.8K 633 511.2K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 356 1023644 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1024000 277 1023723 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 262144 404 261740 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 262144 403 261741 1% /mnt/lustre[OST:1] filesystem_summary: 524114 633 523481 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 1024000 356 1023644 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1024000 277 1023723 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 262144 404 261740 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 262144 403 261741 1% /mnt/lustre2[OST:1] filesystem_summary: 524114 633 523481 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2028 1269276 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 31704 3575316 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 8044 3598976 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 39748 7174292 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2028 1269276 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 1788 1269516 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 31704 3575316 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 8044 3598976 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 39748 7174292 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 10:38:54 (1780324734) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 10:38:57 (1780324737) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 10:39:00 (1780324740) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] PASS 25b (3s) == sanityn test 26a: allow mtime to get older ============ 10:39:03 (1780324743) utime: good mknod times 1780324742 <= 1780324743 <= 1780324743 for /mnt/lustre/f26a utime: good mknod times 1780324742 <= 1780324743 <= 1780324743 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 10:39:07 (1780324747) PASS 26b (5s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 10:39:12 (1780324752) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289279619 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-06-01 10:39:12.000000000 -0400 Birth: 2026-06-01 10:39:12.000000000 -0400 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289279619 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-06-01 10:39:12.000000000 -0400 Birth: 2026-06-01 10:39:12.000000000 -0400 PASS 26c (3s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 10:39:16 (1780324756) striped dir -i0 -c2 -H crush /mnt/lustre/d30.sanityn 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 10:39:21 (1780324761) striped dir -i1 -c2 -H crush2 /mnt/lustre/d31a.sanityn fail_loc=0x314 PASS 31a (4s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 10:39:25 (1780324765) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg437-server mds-ost sync done. Waiting for MDT destroys to complete striped dir -i1 -c2 -H all_char /mnt/lustre/d31b.sanityn fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.137@tcp:/lustre 7666232 40872 7173140 1% /mnt/lustre2 PASS 31b (14s) == sanityn test 31r: open-rename(replace) race =========== 10:39:39 (1780324779) -rw-r--r-- 1 root root 0 Jun 1 10:39 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (6s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 10:39:45 (1780324785) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 47717 Terminated tail -f $tfile PASS 31s (4s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 10:39:49 (1780324789) PASS 31t (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 10:39:53 (1780324793) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 10:39:55 (1780324795) Stopping /mnt/lustre-mds1 (opts:) on oleg437-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg437-server: oleg437-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg437-client: oleg437-server: ssh exited with exit code 1 Started lustre-MDT0000 create file on 2nd stripe under striped directory /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 2 /mnt/lustre/d33c.sanityn/striped touch /mnt/lustre2/d33c.sanityn/striped/subfile SoLC count 2 Below operations shouldn't trigger Solc: create file on 2nd stripe after setattr chmod 777 /mnt/lustre/d33c.sanityn/striped; touch /mnt/lustre2/d33c.sanityn/striped/subfile create remote dir and local dir /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33c.sanityn/remote create local dir after remote dir creation transaction commit /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33c.sanityn/remote.3; do_facet mds2 /home/green/git/lustre-release/lustre/utils/lctl set_param -n osd*.*MDT0001.force_sync 1; mkdir /mnt/lustre/d33c.sanityn/local.3 Solc count 0 PASS 33c (22s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 10:40:18 (1780324818) remote directory create /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre2/d33d.sanityn/subdir CoS count 2 cross-MDT rename mkdir /mnt/lustre/d33d.sanityn; /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn/d1; /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 1 /mnt/lustre/d33d.sanityn/d2; touch /mnt/lustre/d33d.sanityn/d1/src mv /mnt/lustre2/d33d.sanityn/d1/src /mnt/lustre2/d33d.sanityn/d2/tgt CoS count 2 migrate /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 /mnt/lustre/d33d.sanityn /home/green/git/lustre-release/lustre/utils/lfs migrate -m 1 /mnt/lustre2/d33d.sanityn CoS count 2 PASS 33d (23s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 10:40:40 (1780324840) plain dir creation mkdir /mnt/lustre2/d33e.sanityn/plain open file and write echo abc > /mnt/lustre2/d33e.sanityn/f33e.sanityn append write echo dfg >> /mnt/lustre2/d33e.sanityn/f33e.sanityn setattr touch /mnt/lustre2/d33e.sanityn/f33e.sanityn file unlink rm /mnt/lustre2/d33e.sanityn/f33e.sanityn plain dir unlink rmdir /mnt/lustre2/d33e.sanityn/plain striped directory creation /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 2 /mnt/lustre2/d33e.sanityn/striped set default LMV to create striped subdir /home/green/git/lustre-release/lustre/utils/lfs setdirstripe -D -c 2 /mnt/lustre/d33e.sanityn striped subdir creation createmany -d /mnt/lustre/d33e.sanityn/subdir 100 total: 100 mkdir in 0.60 seconds: 167.42 ops/second sub file creation and write createmany -o /mnt/lustre/d33e.sanityn/subfile 100; echo abc > /mnt/lustre/d33e.sanityn/subfile1 total: 100 open/close in 0.64 seconds: 155.14 ops/second sub file append write echo dfg >> /mnt/lustre2/d33e.sanityn/subfile2 subdir setatttr touch /mnt/lustre2/d33e.sanityn/subdir1 subdir unlink unlinkmany -d /mnt/lustre/d33e.sanityn/subdir 100 - unlinked 0 (time 1780324843 ; total 0 ; last 0) total: 100 unlinks in 2 seconds: 50.000000 unlinks/second sub file unlink unlinkmany /mnt/lustre2/d33e.sanityn/subfile 100 - unlinked 0 (time 1780324846 ; total 0 ; last 0) total: 100 unlinks in 0 seconds: inf unlinks/second sub file creation follows striped dir chmod chmod 777 /mnt/lustre/d33e.sanityn/striped; touch /mnt/lustre/d33e.sanityn/striped/subfile striped directory unlink rm -rf /mnt/lustre2/d33e.sanityn/striped directory unlink rm -rf /mnt/lustre2/d33e.sanityn CoS count 0 PASS 33e (9s) == sanityn test 34: no lock timeout under IO ============= 10:40:49 (1780324849) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a4f88538800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9a4f88538800.ost_server_uuid in FULL state after 0 sec oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9a4f88538800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9a4f88538800.ost_server_uuid in IDLE state after 0 sec oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a4f88538800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9a4f88538800.ost_server_uuid in FULL state after 0 sec oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9a4f88538800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9a4f88538800.ost_server_uuid in IDLE state after 0 sec oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a4f88538800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9a4f88538800.ost_server_uuid in IDLE state after 0 sec oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9a4f88538800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9a4f88538800.ost_server_uuid in IDLE state after 0 sec PASS 34 (94s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 10:42:23 (1780324943) striped dir -i1 -c2 -H crush /mnt/lustre/d35.sanityn fail_loc=0x80000317 Race attempt 0 Wait for 58466 58594 for 60 sec... - open/close 1733 (time 1780324956.40 total 10.00 last 173.28) - open/close 3428 (time 1780324966.40 total 20.00 last 169.48) total: 4000 open/close in 23.60 seconds: 169.46 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.137@tcp:/lustre 7.4G 40M 6.9G 1% /mnt/lustre 192.168.204.137@tcp:/lustre 7.4G 40M 6.9G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 10:43:29 (1780325009) striped dir -i0 -c2 -H all_char /mnt/lustre/d36.sanityn Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg437-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.672368 s, 78.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(0) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.561496 s, 93.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(1) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.602247 s, 87.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(2) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.561966 s, 93.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(3) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.70053 s, 74.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(4) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.568924 s, 92.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(5) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.761137 s, 68.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(6) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.750589 s, 69.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(7) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.663482 s, 79.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(8) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.560955 s, 93.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(9) *** before(7173108) after_dd(7121908) after(7173108) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.589525 s, 88.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Waiting for MDT destroys to complete *** cycle(10) *** before(7173108) after_dd(7121908) after(7173108) PASS 36 (205s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 10:46:54 (1780325214) striped dir -i1 -c2 -H crush2 /mnt/lustre/d37.sanityn multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 - create 2749 (time 1780325226.58 total 10.00 last 274.88) - create 5718 (time 1780325236.58 total 20.00 last 296.87) - create 9874 (time 1780325246.58 total 30.00 last 415.48) total: 10000 create in 30.41 seconds: 328.88 ops/second PASS 37 (36s) == sanityn test 39a: file mtime does not change after rename ========================================================== 10:47:30 (1780325250) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 10:47:34 (1780325254) repeat after cancel_lru_locks PASS 39b (5s) == sanityn test 39c: check truncate mtime update ================================================================================ 10:47:38 (1780325258) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 10:47:42 (1780325262) fail_loc=0x411 fail_loc=0 PASS 39d (4s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 10:47:46 (1780325266) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (11s) == sanityn test 40c: pdirops: link and others ======================================================================== 10:47:57 (1780325277) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (9s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 10:48:06 (1780325286) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=10 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (9s) == sanityn test 40e: pdirops: rename and others ======================================================================== 10:48:15 (1780325295) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (9s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 10:48:24 (1780325304) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (7s) == sanityn test 41b: pdirops: create vs create ======================================================================== 10:48:31 (1780325311) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 10:48:38 (1780325318) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (8s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 10:48:45 (1780325325) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41d (7s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 10:48:52 (1780325332) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 41e (7s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 10:49:00 (1780325340) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41f (7s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 10:49:08 (1780325348) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41g (7s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 10:49:15 (1780325355) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 2318: lctl set_param -n fail_loc=0 2>/dev/null || true: No such file or directory Conflict PASS 41h (8s) == sanityn test 41i: reint_open: create vs create ======== 10:49:23 (1780325363) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (777s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 11:02:20 (1780326140) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (6s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 11:02:26 (1780326146) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 11:02:31 (1780326151) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (6s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 11:02:37 (1780326157) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 11:02:43 (1780326163) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (6s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 11:02:48 (1780326168) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42f (6s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 11:02:54 (1780326174) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 11:02:59 (1780326179) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 11:03:04 (1780326184) PASS 43a (31s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 11:03:35 (1780326215) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 11:03:40 (1780326220) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 43c (6s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 11:03:46 (1780326226) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 11:03:52 (1780326232) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 11:03:57 (1780326237) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (6s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 11:04:03 (1780326243) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 11:04:08 (1780326248) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43h (6s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 11:04:14 (1780326254) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43i (5s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 11:04:19 (1780326259) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (39s) == sanityn test 43k: unlink vs create ==================== 11:04:58 (1780326298) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (434s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 11:12:12 (1780326732) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon Jun 1 11:12:15 EDT 2026 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 11:12:17 (1780326737) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 11:12:22 (1780326742) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 11:12:26 (1780326746) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 11:12:31 (1780326751) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 11:12:36 (1780326756) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44f (6s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 11:12:42 (1780326762) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44g (8s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 11:12:50 (1780326770) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44h (8s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 11:12:58 (1780326778) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f44i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f44i.sanityn': File exists Conflict PASS 44i (8s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 11:13:06 (1780326786) PASS 45a (112s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 11:14:58 (1780326898) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45b (8s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 11:15:06 (1780326906) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45c (8s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 11:15:14 (1780326914) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (8s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 11:15:22 (1780326922) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45e (9s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 11:15:31 (1780326931) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (8s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 11:15:40 (1780326940) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (9s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 11:15:49 (1780326949) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45h (8s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 11:15:57 (1780326957) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45i (8s) == sanityn test 45j: read vs rename ====================== 11:16:05 (1780326965) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (800s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 11:29:25 (1780327765) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (6s) == sanityn test 46b: pdirops: link vs create ======================================================================== 11:29:31 (1780327771) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 open: File exists Conflict PASS 46b (7s) == sanityn test 46c: pdirops: link vs link ======================================================================== 11:29:38 (1780327778) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (8s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 11:29:46 (1780327786) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46d (7s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 11:29:53 (1780327793) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 Conflict PASS 46e (9s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 11:30:02 (1780327802) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46f (7s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 11:30:09 (1780327809) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 46g (8s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 11:30:17 (1780327817) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 Conflict PASS 46h (7s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 11:30:25 (1780327825) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f46i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f46i.sanityn': File exists Conflict PASS 46i (14s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 11:30:38 (1780327838) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f47a.sanityn': File exists Conflict PASS 47a (19s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 11:30:58 (1780327858) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 47b (17s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 11:31:15 (1780327875) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f47c.sanityn' to '/mnt/lustre2/f47c.sanityn-2': File exists Conflict PASS 47c (18s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 11:31:33 (1780327893) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47d (22s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 11:31:55 (1780327915) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f47e.sanityn' with non-directory Conflict PASS 47e (23s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 11:32:18 (1780327938) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47f (17s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 11:32:36 (1780327956) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47g (19s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 11:32:55 (1780327975) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0554471 s, 185 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (15s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 11:33:10 (1780327990) 0+1 records in 0+1 records out 158 bytes copied, 0.0807256 s, 2.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3581: kill: (239741) - No such process PASS 51a (11s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 11:33:21 (1780328001) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0787802 s, 13.0 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907448 s, 11.6 MB/s 3145728 PASS 51b (20s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 11:33:41 (1780328021) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0240241 s, 42.6 kB/s PASS 51c (12s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 11:33:53 (1780328033) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332693 s, 31.5 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (8s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 11:34:01 (1780328041) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1318 0x526 0x2c0000401 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1318 0x526 0x2c0000401 read lease(1) has applied. PASS 51e (9s) == sanityn test 54: rename locking ======================= 11:34:10 (1780328050) llite.lustre-ffff9a4f88538800.xattr_cache=1 llite.lustre-ffff9a4f8b96c800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762670 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:34:12.000000000 -0400 Modify: 2026-06-01 11:34:12.000000000 -0400 Change: 2026-06-01 11:34:12.000000000 -0400 Birth: 2026-06-01 11:34:11.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281285 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:34:18.000000000 -0400 Modify: 2026-06-01 11:34:20.000000000 -0400 Change: 2026-06-01 11:34:20.000000000 -0400 Birth: 2026-06-01 11:34:18.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762673 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:34:25.000000000 -0400 Modify: 2026-06-01 11:34:25.000000000 -0400 Change: 2026-06-01 11:34:25.000000000 -0400 Birth: 2026-06-01 11:34:25.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281286 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:34:25.000000000 -0400 Modify: 2026-06-01 11:34:26.000000000 -0400 Change: 2026-06-01 11:34:26.000000000 -0400 Birth: 2026-06-01 11:34:24.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129603798762674 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:34:31.000000000 -0400 Modify: 2026-06-01 11:34:33.000000000 -0400 Change: 2026-06-01 11:34:33.000000000 -0400 Birth: 2026-06-01 11:34:31.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 4096 Blocks: 8 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205289281288 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:34:32.000000000 -0400 Modify: 2026-06-01 11:34:33.000000000 -0400 Change: 2026-06-01 11:34:33.000000000 -0400 Birth: 2026-06-01 11:34:31.000000000 -0400 PASS 54 (33s) == sanityn test 55a: rename vs unlink target dir ========= 11:34:43 (1780328083) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (13s) == sanityn test 55b: rename vs unlink source dir ========= 11:34:56 (1780328096) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (13s) == sanityn test 55c: rename vs unlink orphan target dir == 11:35:09 (1780328109) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (20s) == sanityn test 55d: rename file vs link ================= 11:35:29 (1780328129) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (15s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 11:35:44 (1780328144) fail_loc=0x155 We survived after AB/BA race test fail_loc=0x0 PASS 55e (20s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 11:36:04 (1780328164) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (22s) == sanityn test 55g: rename: race with trylock =========== 11:36:27 (1780328187) mdt.lustre-MDT0000.enable_rename_trylock=1 mdt.lustre-MDT0001.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre/d55g.sanityn/a1/a2' to a subdirectory of itself, '/mnt/lustre/d55g.sanityn/b1/b2/b3/b4/a2' We survived race with trylock test fail_loc=0x0 PASS 55g (29s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 11:36:55 (1780328215) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1780328218 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 2.65387 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (35s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 11:37:30 (1780328250) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1780328255 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 1.17379 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (115s) == sanityn test 60: Verify data_version behaviour ======== 11:39:25 (1780328365) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d60.sanityn PASS 60 (7s) == sanityn test 70a: cd directory && rm directory ======== 11:39:33 (1780328373) PASS 70a (6s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 11:39:39 (1780328379) PASS 70b (7s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 11:39:46 (1780328386) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00730115 s, 9.0 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0112802 s, 5.8 MB/s File: /mnt/lustre2/f71a.sanityn Size: 262144 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289281328 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:39:47.000000000 -0400 Modify: 2026-06-01 11:39:48.000000000 -0400 Change: 2026-06-01 11:39:48.000000000 -0400 Birth: 2026-06-01 11:39:47.000000000 -0400 33908 extent 0 in offset 65536, length 65536 flags (0x80000000): NET extent 1 in offset 196608, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 2, sum of lengths 131072, expected sum 131072 33909 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0179903 s, 3.6 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0100785 s, 6.5 MB/s File: /mnt/lustre/f71a.sanityn Size: 262144 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289281328 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 11:39:47.000000000 -0400 Modify: 2026-06-01 11:39:48.000000000 -0400 Change: 2026-06-01 11:39:48.000000000 -0400 Birth: 2026-06-01 11:39:47.000000000 -0400 33911 extent 0 in offset 65536, length 65536 flags (0x80000000): NET extent 1 in offset 196608, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 2, sum of lengths 131072, expected sum 131072 33909 PASS 71a (6s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 11:39:52 (1780328392) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0102606 s, 6.4 MB/s extent 0 in offset 0, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 1, sum of lengths 65536, expected sum 65536 PASS 71b (5s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 11:39:57 (1780328397) filefrag 1.47.3-wc2 (11-Nov-2025) check last flag for file with 291 extents 290: 296960.. 296963: 755712.. 755715: 4: 0001: last,net,eof check last flag for file with 292 extents 291: 297984.. 297987: 510976.. 510979: 4: 0000: last,net,eof check last flag for file with 293 extents 292: 299008.. 299011: 692224.. 692227: 4: 0001: last,net,eof PASS 71c (40s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 11:40:37 (1780328437) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0212352 s, 3.1 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0119887 s, 5.5 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0160307 s, 4.1 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0749409 s, 875 kB/s disk usage: 262144 /mnt/lustre/f71d.sanityn file size: 458752 /mnt/lustre/f71d.sanityn extent 0 in offset 0, length 65536 flags (0x80000000): NET extent 1 in offset 131072, length 65536 flags (0x80000000): NET extent 2 in offset 262144, length 65536 flags (0x80000000): NET extent 3 in offset 393216, length 65536 flags (0x80000001): LAST NET No unwritten extents, extents number 4, sum of lengths 262144, expected sum 262144 PASS 71d (61s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 11:41:38 (1780328498) llite.lustre-ffff9a4f88538800.xattr_cache=1 llite.lustre-ffff9a4f8b96c800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (8s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 11:41:47 (1780328507) llite.lustre-ffff9a4f88538800.xattr_cache=1 llite.lustre-ffff9a4f8b96c800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (7s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 11:41:55 (1780328515) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0205157 s, 49.9 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0166741 s, 61.4 kB/s 255310: get lock2 255311: get lock1 255310: done 255310: get lock1 255311: done 255311: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 255311: cannot get lock2: Resource deadlock avoided 255311: exit rc=0 255310: done 255310: exit rc=0 PASS 74 (12s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 11:42:07 (1780328527) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0400131 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315709 s, 33.2 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4264: 255900 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (12s) == sanityn test 76: Verify MDT open_files listing ======== 11:42:20 (1780328540) striped dir -i0 -c2 -H crush /mnt/lustre/d76.sanityn opening files: ................ closing files: ................ PASS 76 (200s) == sanityn test 77a: check FIFO NRS policy =============== 11:45:39 (1780328739) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.56496 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884545 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127743 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140743 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167008 s, 6.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183238 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242705 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340782 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300599 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356257 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349242 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379834 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400232 s, 2.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.464518 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.444111 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.460262 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.500901 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00265334 s, 395 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00244755 s, 428 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147959 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110535 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170509 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140413 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190975 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244977 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138423 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107025 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758301 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11536 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621105 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556827 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0528738 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273575 s, 38.3 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (12s) == sanityn test 77b: check CRR-N NRS policy ============== 11:45:51 (1780328751) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.349752 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435198 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978915 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480015 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815115 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537415 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134324 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156094 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891667 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937441 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116739 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317553 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.342885 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352126 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292654 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.327921 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.484161 s, 2.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685826 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531766 s, 19.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104312 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799408 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098347 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605999 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781851 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563874 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794401 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204511 s, 51.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706626 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758577 s, 13.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342613 s, 30.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840895 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087204 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885068 s, 11.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.753045 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110693 s, 9.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191718 s, 5.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238252 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117926 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185298 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386015 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209881 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.433776 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172919 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40389 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.461315 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237045 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226891 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.412323 s, 2.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379345 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113842 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863326 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00279834 s, 375 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277193 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232953 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155969 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251543 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167277 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133419 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28102 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223554 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10285 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099107 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879278 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665167 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514164 s, 20.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (20s) == sanityn test 77c: check ORR NRS policy ================ 11:46:11 (1780328771) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.293922 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478742 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557747 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224785 s, 4.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211538 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237724 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240734 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21557 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260749 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321344 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344488 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321545 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349797 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319694 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317391 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905636 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127254 s, 82.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.350244 s, 3.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560336 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670664 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.392457 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786981 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140859 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846519 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650906 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933316 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399643 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00450168 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00510289 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112783 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524087 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00354294 s, 296 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366196 s, 28.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.692497 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10367 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117102 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176419 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209427 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228923 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284478 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303809 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28077 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607321 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252446 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34458 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00230572 s, 455 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696841 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012118 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32469 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376608 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00516014 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00248134 s, 423 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343744 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324488 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353648 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.410178 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178589 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160307 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139091 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158499 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137498 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140423 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239735 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145921 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511994 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607491 s, 17.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.777376 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106936 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113837 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553405 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185697 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146889 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100649 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168448 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191513 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170832 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356816 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226532 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287802 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216015 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301236 s, 3.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.410219 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.408003 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00246294 s, 426 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229574 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00257083 s, 408 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00250029 s, 419 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158844 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134468 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136549 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208275 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13904 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130218 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211042 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103241 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216193 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997148 s, 10.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934964 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658635 s, 15.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (19s) == sanityn test 77d: check TRR nrs policy ================ 11:46:30 (1780328790) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.648327 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143572 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175694 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195106 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17054 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173468 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105762 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123495 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113052 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179889 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255802 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.442966 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34966 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.436362 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.368114 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371545 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.539891 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236822 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241243 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273111 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270048 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115001 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102102 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120956 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067821 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535956 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135193 s, 77.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549248 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511206 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466318 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465864 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385402 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333429 s, 31.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.41647 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719354 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185837 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159481 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194936 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180094 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184371 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153552 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22084 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182151 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253984 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22805 s, 4.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221696 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209677 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244943 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258476 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283755 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206624 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0567074 s, 18.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394233 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.046013 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412938 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381368 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0452351 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042242 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0443392 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540154 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04605 s, 22.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820771 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497763 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561547 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072667 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821127 s, 12.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.427483 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823886 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998737 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12151 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217632 s, 4.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251148 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254557 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279822 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248234 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228891 s, 4.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247784 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258672 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250382 s, 4.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236932 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260341 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239175 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344809 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690766 s, 152 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165889 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077961 s, 13.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944012 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125489 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170729 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104994 s, 99.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109942 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708027 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663896 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06184 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178298 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060564 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562819 s, 18.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370447 s, 28.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488936 s, 21.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (15s) == sanityn test 77e: check TBF NID nrs policy ============ 11:46:46 (1780328806) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.37@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.593864 s, 28.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989297 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162641 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145075 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922145 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142055 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912899 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224445 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207454 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105337 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272328 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152379 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.289877 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244968 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15057 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318621 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287166 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154908 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177912 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764531 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809368 s, 13.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819565 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843186 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874771 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518605 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093466 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173232 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048942 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144979 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109224 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148872 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183707 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190291 s, 5.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.575722 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897055 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114121 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481974 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656223 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138069 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119939 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165238 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156935 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188078 s, 5.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224708 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2869 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251556 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269837 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27515 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330089 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266756 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0750072 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505922 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601194 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649145 s, 16.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574101 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683985 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701772 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662955 s, 15.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094907 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0695815 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740431 s, 14.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738632 s, 14.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742674 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686007 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0783855 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958927 s, 10.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=5 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.499781 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131594 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148207 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177978 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227394 s, 4.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228982 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221045 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204747 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159837 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278947 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122663 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230372 s, 4.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310126 s, 3.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264734 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.368716 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267505 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303694 s, 3.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546172 s, 19.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368735 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398681 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103578 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466325 s, 22.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821921 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936602 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110395 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120323 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627488 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089242 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116826 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853386 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140056 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828568 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986482 s, 10.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=7 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.506174 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723674 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584728 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809988 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075141 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211617 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18205 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157901 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318016 s, 3.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268672 s, 3.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297852 s, 3.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333178 s, 3.1 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312698 s, 3.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306022 s, 3.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346784 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337317 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.366331 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107643 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00423048 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00287724 s, 364 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948111 s, 11.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849424 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863706 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0984142 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706337 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100773 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848784 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104355 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992611 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033 s, 31.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026557 s, 39.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140101 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019043 s, 55.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (23s) == sanityn test 77f: check TBF JobID nrs policy ========== 11:47:08 (1780328828) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.505358 s, 33.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116969 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102825 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108853 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942818 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181167 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195978 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177254 s, 5.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206404 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214241 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233705 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200664 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232778 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238733 s, 4.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25313 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238608 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257981 s, 4.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580402 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562552 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05965 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279937 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305467 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332415 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644063 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755926 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0491183 s, 21.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398867 s, 26.3 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552726 s, 19.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554798 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455001 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784938 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479201 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699432 s, 15.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=6 removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.395319 s, 42.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725132 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863736 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103471 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991517 s, 10.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116466 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735212 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126596 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123353 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10678 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18144 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189181 s, 5.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179596 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19344 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167038 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223934 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226196 s, 4.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650624 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647405 s, 16.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727 s, 14.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.054879 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586607 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623481 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440703 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481925 s, 21.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560455 s, 18.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0778651 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930043 s, 11.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805117 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649232 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861041 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0876628 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101073 s, 10.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=4 removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.497683 s, 33.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044472 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747352 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102758 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172123 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188855 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180184 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198997 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209383 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17234 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173656 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22509 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335269 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30121 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320843 s, 3.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356002 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372579 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113851 s, 92.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393698 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665461 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318978 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840534 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775574 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379378 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509818 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941315 s, 11.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515857 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566615 s, 18.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983807 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604454 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109234 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116532 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777889 s, 13.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.485551 s, 34.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621004 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163788 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110005 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142576 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141697 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179179 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201892 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204307 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174045 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18886 s, 5.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230421 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224829 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249299 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203119 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239425 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264067 s, 4.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070513 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0653349 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198311 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187556 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646419 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104775 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1608 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111431 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065651 s, 16.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116581 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582597 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470007 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131351 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420561 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269466 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359603 s, 29.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (20s) == sanityn test 77g: Change TBF type directly ============ 11:47:29 (1780328849) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (8s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 11:47:38 (1780328858) oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg437-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg437-client: oleg437-server: ssh exited with exit code 2 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg437-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg437-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg437-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 PASS 77h (9s) == sanityn test 77i: Change rank of TBF rule ============= 11:47:47 (1780328867) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (18s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 11:48:05 (1780328885) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.476691 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06481 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723195 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848811 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125063 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251897 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215756 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222274 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190127 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247711 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254289 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245245 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241292 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205295 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227487 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209827 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240144 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033889 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00446671 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432578 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225801 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675056 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571805 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925955 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574407 s, 18.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101356 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052156 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074859 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763308 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806102 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723782 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807564 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875729 s, 12.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.04334 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5496 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (51s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 11:48:56 (1780328936) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.344515 s, 48.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702871 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815663 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0951389 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135148 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134577 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135125 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13052 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150186 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209642 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176238 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186401 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200949 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194041 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158146 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180678 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181538 s, 5.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472818 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559752 s, 18.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557868 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515594 s, 20.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.058376 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644052 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556033 s, 18.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626814 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591136 s, 17.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062219 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559487 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060364 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632084 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592037 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588206 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664397 s, 15.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7515 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9053 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.298899 s, 56.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892737 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949151 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976498 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10243 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115848 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118947 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149364 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140225 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160356 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184041 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173938 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206165 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220744 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217678 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243142 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236214 s, 4.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459798 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361062 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038738 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041695 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510238 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445722 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04787 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506782 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514388 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537634 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0688324 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712806 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590325 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0611517 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744413 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807621 s, 13.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6999 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9445 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (118s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 11:50:54 (1780329054) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.301489 s, 55.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771577 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0950808 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954452 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104626 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099274 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105411 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110443 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137211 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11865 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186861 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196972 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181177 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183889 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216679 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186665 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215741 s, 4.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408539 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410908 s, 25.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0469989 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499291 s, 21.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0513721 s, 20.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312139 s, 33.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546812 s, 19.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472984 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475107 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403934 s, 26.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0530704 s, 19.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504248 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580518 s, 18.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503951 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578762 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604743 s, 17.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7055 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9627 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg437-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.300829 s, 55.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343484 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0451699 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652191 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445389 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497275 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074488 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752916 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821069 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102641 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112934 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178905 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174186 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176301 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187476 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186983 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187797 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258133 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250849 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252843 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286841 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314337 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271166 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376085 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264489 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00430757 s, 243 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237998 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039014 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428423 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354082 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309777 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336878 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105313 s, 99.6 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7029 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9627 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (116s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 11:52:50 (1780329170) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.219623 s, 76.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331443 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342969 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153993 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181557 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176887 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182602 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175746 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184948 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172341 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177843 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197484 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190343 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193624 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175573 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164678 s, 6.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173527 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295108 s, 35.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026871 s, 39.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267545 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194774 s, 53.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269208 s, 39.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020413 s, 51.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020522 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127679 s, 82.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288576 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637008 s, 165 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026782 s, 39.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121244 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272616 s, 38.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=8 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=4 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274544 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381201 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00245028 s, 428 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94436 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79531 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.37@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.37@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.188529 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505536 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719165 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690049 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692283 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826844 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730958 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0767697 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698541 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117143 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091208 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123538 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118364 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123259 s, 8.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146728 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128013 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147635 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178013 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205931 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350959 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221789 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344552 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024989 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325092 s, 32.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305696 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345175 s, 30.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298051 s, 35.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352511 s, 29.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352897 s, 29.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335669 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301371 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328929 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0373954 s, 28.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=2 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95786 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77276 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.37@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.213429 s, 78.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388743 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484686 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625155 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559567 s, 18.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101324 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976549 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0902245 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121096 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102272 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110433 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976997 s, 10.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142354 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151211 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144925 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154843 s, 6.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161151 s, 6.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199716 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202281 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320475 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772223 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134133 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205787 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233521 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262797 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237897 s, 44.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306243 s, 34.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338188 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839446 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313846 s, 33.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296392 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851787 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00345054 s, 304 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91685 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89933 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.169692 s, 98.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389501 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377045 s, 27.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132963 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156246 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148606 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154244 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142241 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142471 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153983 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134908 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160079 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166724 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173889 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163907 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177033 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15729 s, 6.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229204 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908055 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112523 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147181 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259332 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136324 s, 76.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147374 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182365 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194608 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206077 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214684 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195008 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230717 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237389 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104976 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310795 s, 33.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7653 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74259 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90167 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76803 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.190127 s, 88.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369853 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411193 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458041 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369851 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833865 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906881 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897162 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795146 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145673 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143125 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121683 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14853 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146783 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150943 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139626 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164501 s, 6.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233162 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182428 s, 57.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211252 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295122 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253947 s, 41.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221999 s, 47.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274605 s, 38.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289858 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261715 s, 40.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235937 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304018 s, 34.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205442 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287573 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02874 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309474 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291894 s, 35.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5084 s, 5.4 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.453 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.185915 s, 90.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369199 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660735 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722137 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985732 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156252 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168035 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157088 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166536 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183698 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188667 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196337 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181876 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184471 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177083 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189886 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199843 s, 5.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=4 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210135 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260703 s, 40.2 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198113 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291265 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247343 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305887 s, 34.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242517 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259217 s, 40.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297084 s, 35.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227638 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308582 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340977 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402879 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274393 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315338 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389348 s, 26.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92781 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7457 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.215778 s, 77.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065448 s, 16.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0783155 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871452 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601967 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114553 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109415 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132354 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154123 s, 6.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163941 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159806 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156259 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161336 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185473 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163877 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18468 s, 5.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194583 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260822 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310101 s, 33.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258518 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286243 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265804 s, 39.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336345 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0390599 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050909 s, 20.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316681 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317756 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541356 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0495953 s, 21.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483692 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517611 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0430256 s, 24.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579231 s, 18.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.80418 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76451 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76965 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76036 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (276s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 11:57:26 (1780329446) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.187398 s, 89.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520255 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643054 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580938 s, 18.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217018 s, 4.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208039 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230069 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236665 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247264 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224606 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23363 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241286 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244703 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258686 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637863 s, 164 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034837 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178125 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224032 s, 46.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209883 s, 50.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264691 s, 4.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361249 s, 29.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382779 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218495 s, 48.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267943 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275563 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241741 s, 43.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195413 s, 53.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193721 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0530277 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244017 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346417 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155725 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280834 s, 3.7 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92479 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76919 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn Waiting for MDT destroys to complete PASS 77kb (32s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 11:57:58 (1780329478) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.37@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.37@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.186359 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103647 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130313 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148594 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12992 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132508 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122979 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117582 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151901 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110529 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141559 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14977 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144251 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156617 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12295 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135424 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17414 s, 6.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615012 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588836 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061203 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692383 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618329 s, 17.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594672 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621676 s, 16.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619118 s, 16.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668667 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710666 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666367 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724859 s, 14.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730581 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.069545 s, 15.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725591 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705073 s, 14.9 MB/s removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92322 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75301 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn Waiting for MDT destroys to complete PASS 77kc (31s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 11:58:29 (1780329509) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.37@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.173258 s, 96.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553699 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424047 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979723 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119282 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863162 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108268 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132608 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119989 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112423 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175246 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153068 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195075 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19091 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2054 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227455 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226103 s, 4.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920451 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865372 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125611 s, 83.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319596 s, 32.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028353 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108058 s, 97.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245621 s, 42.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578786 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117306 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590101 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180283 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00463272 s, 226 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00427559 s, 245 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204907 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516239 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00476545 s, 220 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91597 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.98457 s, 21.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn Waiting for MDT destroys to complete PASS 77kd (26s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 11:58:55 (1780329535) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.185112 s, 90.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906649 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164241 s, 6.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154858 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145653 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169984 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177432 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171415 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211851 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201944 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17657 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220767 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234685 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213222 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207324 s, 5.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237229 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212379 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366193 s, 28.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534017 s, 19.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424846 s, 24.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039835 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0451507 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414611 s, 25.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523287 s, 20.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573309 s, 18.3 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516426 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517661 s, 20.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667611 s, 15.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606464 s, 17.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623729 s, 16.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535871 s, 19.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070419 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682305 s, 15.4 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77687 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76151 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92038 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77694 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn Waiting for MDT destroys to complete PASS 77ke (60s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 11:59:55 (1780329595) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.172628 s, 97.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299368 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381104 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120516 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105445 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115631 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969282 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120966 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113841 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116867 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113166 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119302 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133357 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141083 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131804 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126249 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134704 s, 7.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194502 s, 53.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235892 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192194 s, 54.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196748 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281215 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025445 s, 41.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256265 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307062 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194552 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259145 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022727 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155856 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284253 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586946 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00248686 s, 422 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00329873 s, 318 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4602 s, 5.4 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9527 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn Waiting for MDT destroys to complete PASS 77kf (52s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 12:00:47 (1780329647) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.16952 s, 99.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107514 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992394 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128107 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171236 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169414 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185409 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150855 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185518 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149951 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168607 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204359 s, 5.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193609 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17906 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182349 s, 5.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218997 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211938 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181277 s, 57.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153058 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239391 s, 43.8 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175122 s, 59.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190578 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018557 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211748 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255202 s, 41.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212212 s, 49.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212739 s, 49.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241739 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223686 s, 46.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265953 s, 39.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248383 s, 42.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240045 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302578 s, 34.7 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90703 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73488 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg437-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.305717 s, 54.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907556 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117051 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166248 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510435 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0589417 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14573 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142227 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161528 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213581 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930802 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241497 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205035 s, 5.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148618 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171722 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179062 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174945 s, 6.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9a4f88538800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284682 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249226 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466774 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042743 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649473 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050876 s, 20.6 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9a4f88538800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06153 s, 17.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601414 s, 17.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224722 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651349 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594856 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063565 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735387 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723297 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656088 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110203 s, 95.1 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.97903 s, 10.5 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.88559 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 15.1212 s, 6.9 MB/s Write runtime is 16 s, speed is 6.250000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.5862 s, 9.9 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn Waiting for MDT destroys to complete PASS 77kg (135s) == sanityn test 77kh: Verify that Project ID support for NRS TBF rule ========================================================== 12:03:02 (1780329782) using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg437-client.virtnet /mnt/lustre (opts:) Stopping client oleg437-client.virtnet /mnt/lustre opts: Stopping clients: oleg437-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg437-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg437-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg437-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg437-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg437-server tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) tune2fs 1.47.3-wc2 (11-Nov-2025) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) mqueue on /dev/mqueue type mqueue (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg437-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.37,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg437-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.37,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg437-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.37,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/64539/testresults/sanityn-ldiskfs-DNE-rocky8.10_x86_64-rocky8.10_x86_64-retry1 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.37,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg437-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg437-server' oleg437-server: oleg437-server.virtnet: executing load_modules_local oleg437-server: Loading modules from /home/green/git/lustre-release/lustre oleg437-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg437-server: detected 4 online CPUs by sysfs oleg437-server: MODOPTS_LIBCFS= oleg437-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg437-server: oleg437-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg437-client: oleg437-server: ssh exited with exit code 1 Started lustre-MDT0000 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg437-server: oleg437-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg437-client: oleg437-server: ssh exited with exit code 1 Started lustre-MDT0001 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg437-server: oleg437-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg437-client: oleg437-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg437-server: oleg437-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg437-client: oleg437-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre Starting client oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre Started clients oleg437-client.virtnet: 192.168.204.137@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 Starting client oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 Started clients oleg437-client.virtnet: 192.168.204.137@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9a4fab8ef800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9a4fb1f8d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9a4fab8ef800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9a4fb1f8d000.idle_timeout=debug disable quota as required ost.OSS.ost_io.nrs_policies=tbf projid ost.OSS.ost_io.nrs_tbf_rule=start ext_proj_rw projid={100} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.484183 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860659 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14826 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115903 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213935 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262313 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203442 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283587 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282607 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309669 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329317 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338634 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295807 s, 3.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9a4fab8ef800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321485 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.377088 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298874 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260272 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141973 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153646 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129638 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115088 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119164 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118557 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12733 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128531 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129976 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13148 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132677 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146554 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142344 s, 7.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15119 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172638 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160734 s, 6.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=2 removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.42188 s, 19.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.34862 s, 19.6 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_proj_rw ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=tbf projid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w projid={100}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r projid={100}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.475247 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877791 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976422 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282656 s, 3.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272696 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304984 s, 3.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285618 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313474 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321025 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295995 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.345747 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318334 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352287 s, 3.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34126 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373178 s, 2.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355599 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.367683 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359314 s, 29.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00220318 s, 476 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113051 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00261629 s, 401 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00244489 s, 429 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4fab8ef800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191515 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136767 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167451 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207227 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271398 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638511 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582662 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114288 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142591 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316596 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508925 s, 20.6 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg437-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg437-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.2194 s, 20.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9845 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77kh (241s) == sanityn test 77ki: Add rule with unsupported TBF type should fail for generic TBF ========================================================== 12:07:03 (1780330023) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start ext_fail projid={100}&opcode={ost_write} rate=20: Invalid argument oleg437-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start ext_fail projid={100}&opcode={ost_write} rate=20': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77ki (23s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 12:07:26 (1780330046) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (14s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 12:07:40 (1780330060) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg437-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.90292 s, 269 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg437-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 41.136 s, 255 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (65s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 12:08:45 (1780330125) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.596273 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929706 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989543 s, 10.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255946 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240313 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260819 s, 4.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282666 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.307344 s, 3.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9a4fab8ef800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303338 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330124 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240858 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287484 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300353 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322566 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315506 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288858 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.350364 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982952 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849024 s, 12.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127996 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096314 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129926 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147931 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160505 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129442 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112467 s, 9.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131685 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132438 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146055 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14914 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140999 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142679 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162196 s, 6.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4fab8ef800.lock_unused_count=4 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.53774 s, 18.9 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.24536 s, 20.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.480127 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613928 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759455 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1216 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141981 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625543 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726034 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128923 s, 8.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848019 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211036 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145128 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290929 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241259 s, 4.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9a4fab8ef800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299404 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359252 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.389812 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.43754 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063939 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719956 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890333 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755869 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804922 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804015 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993583 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771494 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989926 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111931 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823443 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919239 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999249 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992364 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107305 s, 9.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9a4fab8ef800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9a4fab8ef800.lock_unused_count=15 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg437-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg437-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.09736 s, 20.6 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.07242 s, 20.7 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) == sanityn test 77o: Changing rank should not panic ====== 12:09:52 (1780330192) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (10s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 12:10:02 (1780330202) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (75s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 12:11:17 (1780330277) mds.MDS.mdt.nrs_policies=tbf oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg437-client: oleg437-server: ssh exited with exit code 36 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg437-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg437-client: oleg437-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (22s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 12:11:39 (1780330299) striped dir -i0 -c1 -H all_char /mnt/lustre/d77r.sanityn mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 299070 kill md_thread xargs PASS 77r (45s) == sanityn test 77s: Check TBF LRU shrinker ============== 12:12:24 (1780330344) jobid_name=%e.%p jobid_var=TEST_77s_JOBVAR mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start touch jobid={touch.*} rate=1000 mds.MDS.mdt.nrs_tbf_rule=start unlink jobid={unlink.*} rate=1000 touch jobid={touch.*} 1000, ref 10 touch jobid={touch.*} 1000, ref 0 unlink jobid={unlink.*} 1000, ref 10 unlink jobid={unlink.*} 1000, ref 0 mds.MDS.mdt.nrs_policies=fifo jobid_var=procname_uid jobid_name=%e.%u PASS 77s (35s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 12:12:59 (1780330379) oleg437-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg437-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg437-client: oleg437-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (6s) == sanityn test 79: xattr: intent error ================== 12:13:05 (1780330385) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d79.sanityn File: /mnt/lustre/d79.sanityn Size: 8192 Blocks: 16 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 162129637319639051 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 12:13:06.000000000 -0400 Modify: 2026-06-01 12:13:06.000000000 -0400 Change: 2026-06-01 12:13:06.000000000 -0400 Birth: 2026-06-01 12:13:06.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (15s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 12:13:20 (1780330400) total: 10 open/close in 0.12 seconds: 86.45 ops/second multiop /mnt/lustre2/d80a.sanityn/f80a.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Finish migration, then checking.. PASS 80a (6s) == sanityn test 80b: Accessing directory during migration ========================================================== 12:13:26 (1780330406) start migration thread 307966 accessing the migrating directory for 1 minute... Waiting for MDT destroys to complete concurrent access 29 failures, 88 successes PASS 80b (116s) == sanityn test 81a: rename and stat under striped directory ========================================================== 12:15:22 (1780330522) stat: cannot statx '/mnt/lustre/d81a.sanityn/d0/0': No such file or directory File: /mnt/lustre/d81a.sanityn/d0/0 Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115272381367980 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 12:15:23.000000000 -0400 Modify: 2026-06-01 12:15:23.000000000 -0400 Change: 2026-06-01 12:15:23.000000000 -0400 Birth: 2026-06-01 12:15:22.000000000 -0400 PASS 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 12:15:24 (1780330524) total: 2500 open/close in 8.81 seconds: 283.80 ops/second /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 309851 Killed ( while true; do touch $DIR1/$tdir; done ) (wd: ~) PASS 81b (186s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 12:18:31 (1780330711) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (3s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 12:18:34 (1780330714) - open/close 699 (time 1780330730.76 total 10.01 last 69.84) - open/close 1447 (time 1780330740.77 total 20.02 last 74.72) - open/close 2109 (time 1780330750.79 total 30.03 last 66.10) total: 2500 open/close in 35.04 seconds: 71.35 ops/second - open/close 760 (time 1780330770.91 total 10.00 last 76.00) - open/close 1441 (time 1780330780.92 total 20.02 last 67.98) - open/close 2168 (time 1780330790.93 total 30.02 last 72.67) total: 2500 open/close in 34.60 seconds: 72.26 ops/second snapshot_time 1780330952.346555654 secs.nsecs start_time 1780330717.159514807 secs.nsecs elapsed_time 235.187040847 secs.nsecs open 2505 samples [usecs] 89 16491 3294698 7777890326 close 2504 samples [usecs] 159 8258 827646 683091374 mknod 2501 samples [usecs] 458 12787 2713080 5536084902 unlink 2500 samples [usecs] 352 72224 5027726 76479872604 mkdir 2 samples [usecs] 2664 34531 37195 1199486857 rmdir 2 samples [usecs] 4624 13432 18056 201800000 rename 2500 samples [usecs] 349 44502 18464208 296158400150 getattr 3765 samples [usecs] 1 14945 313819 987068011 statfs 46 samples [usecs] 10 2122 2921 4518161 parallel_rename_file 2500 samples [usecs] 349 44502 18464208 296158400150 crossdir_rename 2500 samples [usecs] 349 44502 18464208 296158400150 snapshot_time 1780330952.346664242 secs.nsecs start_time 1780330717.159616814 secs.nsecs elapsed_time 235.187047428 secs.nsecs open 2504 samples [usecs] 137 50051 3697396 11965290528 close 2504 samples [usecs] 149 14647 884419 923590087 mknod 2500 samples [usecs] 515 40821 3045170 8187135026 unlink 2500 samples [usecs] 367 126606 4147058 40491326550 mkdir 2 samples [usecs] 1662 40970 42632 1681303144 rmdir 2 samples [usecs] 19127 20144 39271 771622865 rename 2500 samples [usecs] 383 42418 13913758 158766959884 getattr 3759 samples [usecs] 10 15252 411639 1087987369 statfs 46 samples [usecs] 9 33 784 14462 parallel_rename_file 2500 samples [usecs] 383 42418 13913758 158766959884 crossdir_rename 2500 samples [usecs] 383 42418 13913758 158766959884 PASS 81d (245s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 12:22:39 (1780330959) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.8207 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.8207 PASS 82 (7s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 12:22:46 (1780330966) start pid 324445 to create/unlink striped directory start pid 324446 to stat striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5987: 324445 Terminated ( cd $DIR1; while true; do $LFS mkdir -i1 -c2 $tdir; rmdir $tdir; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5987: 324446 Terminated ( cd $DIR2; while true; do stat $tdir > /dev/null 2>&1; done ) PASS 83 (129s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 12:24:55 (1780331095) fail_loc=0x8000060b PASS 84 (15s) == sanityn test 85: Lustre API root cache race =========== 12:25:10 (1780331110) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1780331112 Finishing test test1 at 1780331118 PASS 85 (13s) == sanityn test 90: open/create and unlink striped directory ========================================================== 12:25:24 (1780331124) start pid 329766 to open/create under striped directory start pid 329767 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6054: 329766 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; touch $tdir/f{0..3} > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6054: 329767 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 90 (188s) == sanityn test 91: chmod and unlink striped directory === 12:28:32 (1780331312) start pid 332360 to chmod striped directory start pid 332362 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6091: 332360 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; chmod go+w $tdir > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6091: 332362 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 91 (188s) == sanityn test 92: create remote directory under orphan directory ========================================================== 12:31:40 (1780331500) fail_loc=0x1408 mkdir: cannot create directory '/mnt/lustre2/d92.sanityn/dir': No such file or directory lfs setdirstripe: cannot resolve path '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory (2) lfs setdirstripe: '/mnt/lustre2/d92.sanityn/remote_dir' is not on a Lustre filesystem: No such file or directory (2) lfs setdirstripe: cannot create dir '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory fail_loc=0 PASS 92 (7s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 12:31:48 (1780331508) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 3586 0xe02 0x2c0000400 0 3586 0xe02 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5786 0x169a 0x280000401 1 5781 0x1695 0x2c0000401 rc1=0 and rc2=0 PASS 93 (16s) == sanityn test 94: signal vs CP callback race =========== 12:32:05 (1780331525) osc.lustre-OST0000-osc-ffff9a4fab8ef800.idle_timeout=0 osc.lustre-OST0000-osc-ffff9a4fb1f8d000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9a4fab8ef800.idle_timeout=0 osc.lustre-OST0001-osc-ffff9a4fb1f8d000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.052518 s, 156 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 2.98051 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6178: 337493 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0344968 s, 119 kB/s osc.lustre-OST0000-osc-ffff9a4fab8ef800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9a4fb1f8d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9a4fab8ef800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9a4fb1f8d000.idle_timeout=debug PASS 94 (15s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 12:32:20 (1780331540) llite.lustre-ffff9a4fab8ef800.fast_read=0 llite.lustre-ffff9a4fb1f8d000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0441454 s, 47.5 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff9a4fab8ef800.fast_read=1 llite.lustre-ffff9a4fb1f8d000.fast_read=1 PASS 95a (20s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 12:32:40 (1780331560) llite.lustre-ffff9a4fab8ef800.fast_read=0 llite.lustre-ffff9a4fb1f8d000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.0172623 s, 712 kB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff9a4fab8ef800.fast_read=1 llite.lustre-ffff9a4fb1f8d000.fast_read=1 PASS 95b (13s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 12:32:53 (1780331573) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 12:32:57 (1780331577) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0212792 s, 192 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (8s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 12:33:05 (1780331585) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.140049 s, 15.0 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (8s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 12:33:13 (1780331593) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.111514 s, 18.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (7s) == sanityn test 100e: DoM: read on open and file size ==== 12:33:20 (1780331600) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00630055 s, 1.9 MB/s PASS 100e (7s) == sanityn test 101a: Discard DoM data on unlink ========= 12:33:27 (1780331607) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.016782 s, 244 kB/s PASS 101a (7s) == sanityn test 101b: Discard DoM data on rename ========= 12:33:34 (1780331614) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0125658 s, 326 kB/s PASS 101b (6s) == sanityn test 101c: Discard DoM data on close-unlink === 12:33:40 (1780331620) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0216994 s, 189 kB/s PASS 101c (8s) == sanityn test 102: Test open by handle of unlinked file ========================================================== 12:33:48 (1780331628) file: /mnt/lustre/f102.sanityn fh_bytes: 32 fh_type: 151 fh_data: a1 13 00 00 02 00 00 00 16 19 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x2000013a1:0x1916:0x0] access via mount point '/mnt/lustre2' - OK access after unlink - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.remote fh_bytes: 32 fh_type: 151 fh_data: d4 0b 00 40 02 00 00 00 42 13 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000bd4:0x1342:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.remote/subdir fh_bytes: 32 fh_type: 151 fh_data: a1 13 00 00 02 00 00 00 17 19 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x2000013a1:0x1917:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.stripe fh_bytes: 32 fh_type: 151 fh_data: d4 0b 00 40 02 00 00 00 43 13 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000bd4:0x1343:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! file: /mnt/lustre/d102.sanityn.stripe/subdir fh_bytes: 32 fh_type: 151 fh_data: d4 0b 00 40 02 00 00 00 44 13 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x240000bd4:0x1344:0x0] access via mount point '/mnt/lustre2' - OK check_fhandle_syscalls test Passed! PASS 102 (8s) == sanityn test 103: Test size correctness with lockahead ========================================================== 12:33:56 (1780331636) striped dir -i1 -c2 -H all_char /mnt/lustre/d103.sanityn fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1780331638 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1780331641 Finishing test test23 at 1780331642 PASS 103 (12s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 12:34:08 (1780331648) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0131202 s, 78.0 kB/s oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1780331651 1780331651 1780331651 MDT a|m|ctime 1780331651 1780331651 1780331651 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00931987 s, 110 kB/s oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1780331651 1780331654 1780331654 MDT a|m|ctime 1780331651 1780331654 1780331654 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0702628 s, 14.6 kB/s oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1780331651 1780331660 1780331660 MDT a|m|ctime 1780331651 1780331660 1780331660 oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1780331651 1780331660 1780331660 MDT a|m|ctime 1780331651 1780331660 1780331660 oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) STAT a|m|ctime 1780331651 1780331660 1780331671 MDT a|m|ctime 1780331651 1780331660 1780331671 PASS 104 (30s) == sanityn test 105: Glimpse and lock cancel race ======== 12:34:39 (1780331679) striped dir -i1 -c2 -H crush /mnt/lustre/d105.sanityn fail_loc=0x416 PASS 105 (23s) == sanityn test 106a: Verify the btime via statx() ======= 12:35:03 (1780331703) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0166753 s, 61.4 kB/s oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) Inode: 5150 Type: regular Mode: 0644 Flags: 0x80000 Generation: 1102315055 Version: 0x00000003:00009a59 User: 0 Group: 0 Project: 0 Size: 0 File ACL: 0 Links: 1 Blockcount: 0 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x6a1db4b9:00000000 -- Mon Jun 1 12:35:05 2026 atime: 0x6a1db4b9:00000000 -- Mon Jun 1 12:35:05 2026 mtime: 0x6a1db4b9:00000000 -- Mon Jun 1 12:35:05 2026 crtime: 0x6a1db4b8:96ae3800 -- Mon Jun 1 12:35:04 2026 Size of extra inode fields: 32 Extended attributes: lma: fid=[0x2000013a1:0x191b:0x0] compat=0 incompat=0 trusted.lov (56) = d0 0b d1 0b 01 00 00 00 1b 19 00 00 00 00 00 00 a1 13 00 00 02 00 00 00 00 00 40 00 01 00 00 00 01 04 00 80 02 00 00 00 a0 16 00 00 00 00 00 00 00 00 00 00 00 00 00 00 user.job (4) = "dd.0" linkea: idx=0 parent=[0x200000007:0x1:0x0] name='f106a.sanityn' trusted.som (24) = 04 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EXTENTS: PASS 106a (8s) == sanityn test 106b: Glimpse RPCs test for statx ======== 12:35:11 (1780331711) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0167667 s, 61.1 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 8 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115272381372700 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 12:35:13.000000000 -0400 Modify: 2026-06-01 12:35:13.000000000 -0400 Change: 2026-06-01 12:35:13.000000000 -0400 Birth: 2026-06-01 12:35:13.000000000 -0400 /mnt/lustre/f106b.sanityn 144115272381372700 -rw-r--r-- 1 0 0 1780331713 1780331713 1780331713 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 8 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115272381372700 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 12:35:13.000000000 -0400 Modify: 2026-06-01 12:35:13.000000000 -0400 Change: 2026-06-01 12:35:13.000000000 -0400 Birth: 2026-06-01 12:35:13.000000000 -0400 1780331713 1024 8 PASS 106b (8s) == sanityn test 106c: Verify statx attributes mask ======= 12:35:19 (1780331719) PASS 106c (7s) == sanityn test 107a: Basic grouplock conflict =========== 12:35:26 (1780331726) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.324937 s, 32.3 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5788 0x169c 0x2c0000401 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 First grouplock blocks second one PASS 107a (9s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 12:35:35 (1780331735) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.319326 s, 32.8 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5794 0x16a2 0x280000401 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.8207 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (13s) == sanityn test 108a: lseek: parallel updates ============ 12:35:48 (1780331748) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00768722 s, 1.3 MB/s llite.lustre-ffff9a4fab8ef800.hybrid_io=0 llite.lustre-ffff9a4fb1f8d000.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.50291 s, 1.9 MB/s PASS 108a (11s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 12:35:59 (1780331759) 192.168.204.137@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg437-client.virtnet /mnt/lustre (opts:) 192.168.204.137@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg437-client.virtnet /mnt/lustre2 (opts:) Starting race between client mounts (duration 30) Iteration 0 Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre 192.168.204.137@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.137@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg437-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg437-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg437-server' oleg437-server: oleg437-server.virtnet: executing load_modules_local oleg437-server: Loading modules from /home/green/git/lustre-release/lustre oleg437-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg437-server: detected 4 online CPUs by sysfs oleg437-server: MODOPTS_LIBCFS= oleg437-server: Force libcfs to create 2 CPU partitions Iteration 1 Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 192.168.204.137@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.137@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg437-client.virtnet /mnt/lustre (opts:) Stopping client oleg437-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg437-server' oleg437-server: oleg437-server.virtnet: executing load_modules_local oleg437-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg437-server: Loading modules from /home/green/git/lustre-release/lustre oleg437-server: detected 4 online CPUs by sysfs oleg437-server: MODOPTS_LIBCFS= oleg437-server: Force libcfs to create 2 CPU partitions Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre Starting client: oleg437-client.virtnet: -o user_xattr,flock 192.168.204.137@tcp:/lustre /mnt/lustre2 PASS 109 (70s) == sanityn test 110: do not grant another lock on resend ========================================================== 12:37:09 (1780331829) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115272431697923 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-01 12:37:10.000000000 -0400 Modify: 2026-06-01 12:37:10.000000000 -0400 Change: 2026-06-01 12:37:10.000000000 -0400 Birth: 2026-06-01 12:37:10.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg437-client.virtnet oleg437-client.virtnet PASS 110 (71s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 12:38:20 (1780331900) fail_loc=0x8000018a /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6977: 353847 Terminated $MULTIOP $DIR2/$tdir/mdt0dir/foodir/file2 Ow4096_c PASS 111 (12s) == sanityn test 112: update max-inherit in default LMV === 12:38:33 (1780331913) lmv.lustre-clilmv-ffff9a4fab8cf800.qos_threshold_rr=90 lmv.lustre-clilmv-ffff9a4fb1ef6800.qos_threshold_rr=90 PASS 112 (14s) == sanityn test 113: check servers of specified fs ======= 12:38:46 (1780331926) lustre-OST0000-osc-ffff9a4fb1ef6800 active. lustre-OST0001-osc-ffff9a4fb1ef6800 active. lustre-OST0000-osc-ffff9a4fb1ef6800 active. lustre-OST0001-osc-ffff9a4fb1ef6800 active. MGC192.168.204.137@tcp active. PASS 113 (6s) == sanityn test 114: implicit default LMV inherit ======== 12:38:52 (1780331932) /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1 lmv_stripe_count: 2 lmv_stripe_offset: 2 lmv_hash_type: none lmv_max_inherit: 2 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1 lmv_stripe_count: 2 lmv_stripe_offset: 2 lmv_hash_type: none lmv_max_inherit: 2 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 2 lmv_stripe_offset: 1 lmv_hash_type: none lmv_max_inherit: 3 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 /mnt/lustre2/d114.sanityn/l1/l2/l3 lmv_stripe_count: 1 lmv_stripe_offset: -1 lmv_hash_type: none lmv_max_inherit: 1 lmv_max_inherit_rr: 0 PASS 114 (26s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 12:39:18 (1780331958) - create 1659 (time 1780331972.38 total 10.00 last 165.85) total: 3000 create in 18.54 seconds: 161.80 ops/second oleg437-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg437-server: htree_dump_int_block: Ext2 directory block not found while calculating checksum for htree root oleg437-server: USE: f2729 f1238 fail_loc=0x80002401 fail_val=5 mkdir: cannot create directory '/mnt/lustre/d115.sanityn/f1238': File exists PASS 115 (36s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 12:39:54 (1780331994) llite.lustre-ffff9a4fab8cf800.intent_mkdir=1 llite.lustre-ffff9a4fb1ef6800.intent_mkdir=1 MD layout /mnt/lustre/d116.sanityn: lmv_stripe_count: 2 lmv_stripe_offset: 0 lmv_hash_type: crush mdtidx FID[seq:oid:ver] 0 [0x2000013a0:0x14e:0x0] 1 [0x240000bd2:0x14a:0x0] mkdir /mnt/lustre/d116.sanityn/tdir0 setdirstripe -D -i1 /mnt/lustre2/d116.sanityn/tdir0 mkdir /mnt/lustre/d116.sanityn/tdir0/tdir11 lmv_stripe_count: 0 lmv_stripe_offset: 0 lmv_hash_type: none lmv_stripe_count: 0 lmv_stripe_offset: 1 lmv_hash_type: none PASS 116 (8s) == sanityn test 117a: TCU: Init and enable Trash Can on MDTs ========================================================== 12:40:02 (1780332002) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 PASS 117a (9s) == sanityn test 117b: Move regular file and empty dir into trash can dir ========================================================== 12:40:11 (1780332011) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain llite.lustre-ffff9a4fab8cf800.trash_file_read=1 llite.lustre-ffff9a4fb1ef6800.trash_file_read=1 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.204803 s, 20.5 MB/s /mnt/lustre/.lustre/.ltrash: MDT0000 MDT0001 /mnt/lustre/.lustre/.ltrash/MDT0000: 0x2000013a4:0xbcc:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0001: llite.lustre-ffff9a4fab8cf800.trash_file_read=0 llite.lustre-ffff9a4fb1ef6800.trash_file_read=0 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117b (15s) == sanityn test 117c: Move deleted tree with multiple levels into trash ========================================================== 12:40:26 (1780332026) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/.ltrash/MDT0000: 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: tdir1 tdir2 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0/tdir1: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0/tdir1/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0/tdir1/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0/tdir2: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117c (15s) == sanityn test 117d: Per-User Trash can Type testing ==== 12:40:41 (1780332041) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid Trash Can Type: uid uid running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2/tdir21] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [mkdir] [/mnt/lustre/d117d.sanityn/tdir2/tdir22] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d117d.sanityn/tdir2/tfile23] uid1: 0 uid2: 500 /mnt/lustre/.lustre/.ltrash/MDT0000: 0 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 500 /mnt/lustre/.lustre/.ltrash/MDT0000/0: 0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: /mnt/lustre/.lustre/.ltrash/MDT0000/500: 0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117d (16s) == sanityn test 117e: Undeleted dir in trash should keep its original xattrs ========================================================== 12:40:57 (1780332057) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/.ltrash/MDT0000: 0 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 0x2000013a4:0xbe3:0x0 500 /mnt/lustre/.lustre/.ltrash/MDT0000/0: 0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/500: 0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: trusted.name11="value11" trusted.name12="value12" trusted.name1="value1" mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117e (14s) == sanityn test 117f: Uncache the dentry under the trash dir ========================================================== 12:41:11 (1780332071) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/.ltrash/MDT0000: 0 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 0x2000013a4:0xbe3:0x0 0x2000013a4:0xbe7:0x0 500 /mnt/lustre/.lustre/.ltrash/MDT0000/0: 0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0: tdir1 tfile3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/500: 0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: tdir1 tfile3 /mnt/lustre/.lustre/.ltrash/MDT0000: 0 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 0x2000013a4:0xbe3:0x0 0x2000013a4:0xbe7:0x0 500 /mnt/lustre/.lustre/.ltrash/MDT0000/0: 0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0: tdir1 tdir2 tfile3 tfile4 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/500: 0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: tdir2 tfile4 -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tf2 -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tf3 -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tf1 -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11: -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tfile12 -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0 /mnt/lustre/.lustre/.ltrash/MDT0000/0: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tfile13 -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/500 /mnt/lustre/.lustre/.ltrash/MDT0000/500: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tfile23 -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir2 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir2: -u-------------------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir1: -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tfile3 -u------------e------- /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tfile4 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117f (17s) == sanityn test 117g: Access .Trash for a non-striped directory ========================================================== 12:41:28 (1780332088) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain /mnt/lustre/.lustre/.ltrash/MDT0000: 0 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 0x2000013a4:0xbe3:0x0 0x2000013a4:0xbe7:0x0 0x2000013a4:0xbec:0x0 500 /mnt/lustre/.lustre/.ltrash/MDT0000/0: 0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0: tdir1 tdir2 tfile3 tfile4 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbec:0x0: tdir1 tfile3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbec:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/500: 0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: ==> check '.Trash': tdir1 tfile3 <== tdir1 tfile3 tdir1 tfile3 tdir21 tfile22 tdir2 tdir21 tfile22 mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117g (17s) == sanityn test 117h: Permission testing of various operations for files in Trash Can ========================================================== 12:41:45 (1780332105) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0001.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain mdd.lustre-MDT0001.trash_can_type=plain Trash Can Type: plain plain Trash list: ===> /mnt/lustre/.lustre/.ltrash/MDT0000: 0 0x2000013a4:0xbcc:0x0 0x2000013a4:0xbd2:0x0 0x2000013a4:0xbe3:0x0 0x2000013a4:0xbe7:0x0 0x2000013a4:0xbec:0x0 0x2000013a4:0xbf3:0x0 500 /mnt/lustre/.lustre/.ltrash/MDT0000/0: 0x2000013a4:0xbdb:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0: tdir11 tdir12 tfile13 /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0/0x2000013a4:0xbdb:0x0/tdir12: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0: tdir1 tdir2 tf1 tf2 tf3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbcc:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbd2:0x0: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0: tdir1 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1: tdir11 tfile12 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe3:0x0/tdir1/tdir11: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0: tdir1 tdir2 tfile3 tfile4 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbe7:0x0/tdir2: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbec:0x0: tdir1 tdir2 tfile3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbec:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbec:0x0/tdir2: tdir21 tfile22 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbec:0x0/tdir2/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0: tdir1 tfile3 /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0/tdir1: /mnt/lustre/.lustre/.ltrash/MDT0000/500: 0x2000013a4:0xbdf:0x0 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0: tdir21 tdir22 tfile23 /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir21: /mnt/lustre/.lustre/.ltrash/MDT0000/500/0x2000013a4:0xbdf:0x0/tdir22: <=== touch: setting times of '/mnt/lustre/.lustre/.ltrash/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/.ltrash/f117h.sanityn.crfail': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/.ltrash/MDT0000/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/.ltrash/MDT0000/f117h.sanityn.crfail': Operation not permitted rmdir: failed to remove '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0/f117h.sanityn.crfail': No such file or directory mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/.ltrash/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/.ltrash/MDT0000/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/.ltrash/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/.ltrash/MDT0000/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0/d117h.sanityn.mvfail': Operation not permitted chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3 chmod: changing permissions of '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3': Bad file descriptor truncate: cannot truncate '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3' to length 4096: Bad file descriptor chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1 chmod: changing permissions of '/mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1': Bad file descriptor setfattr: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1: Bad file descriptor setfattr: /mnt/lustre/.lustre/.ltrash/MDT0000/0x2000013a4:0xbf3:0x0//tdir1: Bad file descriptor open: Bad file descriptor open: Bad file descriptor read: Operation not permitted mmap: Operation not permitted open: Is a directory open: Is a directory mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0001.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid mdd.lustre-MDT0001.trash_can_type=uid PASS 117h (14s) == sanityn test 121: trunc append race =================== 12:41:59 (1780332119) 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0257526 s, 795 kB/s fail_loc=0x80001436 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0655539 s, 312 kB/s 0000000 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0005000 0000 0000 0000 0000 0000 0000 0000 0000 * 000c800 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0011800 PASS 121 (8s) == sanityn test 200: service remains healthy while able to process request ========================================================== 12:42:07 (1780332127) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=5 striped dir -i0 -c2 -H all_char /mnt/lustre/d200.sanityn fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0759 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0653 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.064 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.0751 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 52.5034 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 52.5177 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2039 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 84.5372 s, 0.0 kB/s1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 84.529 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 84.5252 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg437-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a4fab8cf800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9a4fab8cf800.ost_server_uuid in IDLE state after 0 sec PASS 200 (103s) cleanup: ====================================================== == sanityn test complete, duration 8939 sec ============== 12:43:52 (1780332232) === sanityn: start cleanup 12:43:53 (1780332233) === Stopping clients: oleg437-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg437-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 12:49:10 (1780332550) ===