-----============= acceptance-small: sanityn ============----- Tue Feb 25 06:59:57 EST 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg153-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 07:00:02 (1740484802) === Starting client oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Started clients oleg153-client.virtnet: 192.168.201.153@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg153-client.virtnet: executing check_config_client /mnt/lustre oleg153-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg153-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b581e800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b58d8000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b581e800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58d8000.idle_timeout=debug disable quota as required oleg153-server: oleg153-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 07:00:09 (1740484809) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134495 s, 78.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f6990] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 07:00:10 (1740484810) PASS 0 (2s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 07:00:12 (1740484812) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 07:00:14 (1740484814) -rw-r--r-- 1 root root 0 Feb 25 07:00 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 07:00:15 (1740484815) -rw-r--r-- 1 root root 0 Feb 25 07:00 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 07:00:17 (1740484817) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 07:00:18 (1740484818) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 07:00:20 (1740484820) total 1 -rwxrwxrwx 1 root root 0 Feb 25 07:00 f2a -rwxrwxrwx 1 root root 0 Feb 25 07:00 f2b total 1 -rwxrwxrwx 1 root root 0 Feb 25 07:00 f2a -rwxrwxrwx 1 root root 0 Feb 25 07:00 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00343939 s, 149 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 07:00:21 (1740484821) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 07:00:22 (1740484822) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0713905 s, 29.4 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 07:00:24 (1740484824) PASS 3 (1s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 07:00:25 (1740484825) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 07:00:28 (1740484828) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 07:00:29 (1740484829) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) == sanityn test 7: remove of open directory on other node ======================================================================= 07:00:30 (1740484830) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 07:00:32 (1740484832) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 07:00:33 (1740484833) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 07:00:35 (1740484835) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102727 s, 102 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0106959 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (1s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 07:00:36 (1740484836) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00164646 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00240364 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0027309 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00248304 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00213702 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00199724 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00198373 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00200322 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00254163 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00233944 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00213001 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0021188 s, 0.5 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 07:00:38 (1740484838) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00299319 s, 1.0 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00630055 s, 650 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000195187 s, 15.7 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 07:00:39 (1740484839) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 07:00:41 (1740484841) warning: '-runtime' deprecated, use '-t runtime' instead - link 2239 (time 1740484851.90 total 10.00 last 223.83) - link 4526 (time 1740484861.90 total 20.00 last 228.67) - link 6665 (time 1740484871.90 total 30.01 last 213.82) - link 8767 (time 1740484881.91 total 40.01 last 210.13) - link 10000 (time 1740484887.09 total 45.20 last 237.67) - link 12687 (time 1740484897.10 total 55.20 last 268.67) - link 15010 (time 1740484907.10 total 65.20 last 232.23) - link 17147 (time 1740484917.10 total 75.21 last 213.57) - link 19216 (time 1740484927.11 total 85.21 last 206.89) - link 20000 (time 1740484931.15 total 89.25 last 193.85) - link 21917 (time 1740484941.15 total 99.26 last 191.67) total: 22081 link in 100.01 seconds: 220.79 ops/second using seed 364414092 running for 100 seconds - stat 10000 (time 1740484848 ; total 6 ; last 6) - stat 20000 (time 1740484853 ; total 11 ; last 5) - stat 30000 (time 1740484858 ; total 16 ; last 5) - stat 40000 (time 1740484863 ; total 21 ; last 5) - stat 50000 (time 1740484869 ; total 27 ; last 6) - stat 60000 (time 1740484874 ; total 32 ; last 5) - stat 70000 (time 1740484879 ; total 37 ; last 5) - stat 80000 (time 1740484884 ; total 42 ; last 5) - stat 90000 (time 1740484890 ; total 48 ; last 6) - stat 100000 (time 1740484898 ; total 56 ; last 8) - stat 107715 (time 1740484909 ; total 67 ; last 11) - stat 110000 (time 1740484912 ; total 70 ; last 3) - stat 117283 (time 1740484923 ; total 81 ; last 11) - stat 120000 (time 1740484926 ; total 84 ; last 3) - stat 126460 (time 1740484937 ; total 95 ; last 11) total: 129521 stats in 100 seconds: 1295.209961 stats/second - unlinked 0 (time 1740484892 ; total 0 ; last 0) - unlinked 10000 (time 1740484952 ; total 60 ; last 60) - unlinked 20000 (time 1740484983 ; total 91 ; last 31) unlink(/mnt/lustre2/lockdir/lockfile22081) error: No such file or directory total: 22080 unlinks in 97 seconds: 227.628860 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21997) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22000) - No such process PASS 12 (150s) == sanityn test 13: test directory page revocation ======= 07:03:11 (1740484991) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 07:03:12 (1740484992) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4866: 23394 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 07:03:14 (1740484994) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4866: 24028 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 07:03:16 (1740484996) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24660 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 07:03:17 (1740484997) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25295 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 07:03:19 (1740484999) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 25929 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 07:03:21 (1740485001) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7520256 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7520256kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (13s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 07:03:34 (1740485014) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.168964 s, 49.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.14095 s, 59.5 MB/s Chance of close/open is 1 in 50 Seed set to 5015 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1740485015.571857 mapwrite 0x1ad3ec thru 0x1bd0eb (0xfd00 bytes) 000004[0] 1740485015.672857 read 0x18b90b thru 0x1969d0 (0xb0c6 bytes) truncating to largest ever: 0x3109d1 000013[0] 1740485015.732468 trunc from 0x1bd0ec to 0x3109d0 (0x1538e5 bytes) 000014[0] 1740485015.738373 mapread 0x15734d thru 0x164f38 (0xdbec bytes) 000016[1] 1740485015.745773 mapread 0x280222 thru 0x28842d (0x820c bytes) 000017[0] 1740485015.763295 mapwrite 0x890903 thru 0x89fd8b (0xf489 bytes) 000018[0] 1740485015.800365 write 0x97ae02 thru 0x9897b8 (0xe9b7 bytes) 000019[0] 1740485015.801687 mapwrite 0x5282d8 thru 0x52cd82 (0x4aab bytes) 000022[1] 1740485015.843203 write 0x136ee9 thru 0x137bca (0xce2 bytes) 000025[1] 1740485015.864285 mapread 0x3cc205 thru 0x3d98e1 (0xd6dd bytes) truncating to largest ever: 0x7228c0 000026[1] 1740485015.886186 trunc from 0x9897b9 to 0x7228bf (0x266ef9 bytes) 000027[0] 1740485015.903256 read 0x61b3a6 thru 0x61c73e (0x1399 bytes) 000028[1] 1740485015.918200 trunc from 0x7228c0 to 0x220c56 (0x501c69 bytes) 000029[0] 1740485015.933711 write 0x439f46 thru 0x441b73 (0x7c2e bytes) 000030[1] 1740485015.947506 trunc from 0x441b74 to 0x2a8a7d (0x1990f6 bytes) 000034[0] 1740485015.965613 mapread 0x0fae9c thru 0x0ff89c (0x4a01 bytes) 000039[0] 1740485016.005726 write 0x139434 thru 0x1422e3 (0x8eb0 bytes) 000041[1] 1740485016.028572 mapread 0x140b03 thru 0x148712 (0x7c10 bytes) 000044[0] 1740485016.052046 trunc from 0x2a8a7e to 0x4b4072 (0x20b5f5 bytes) 000046[0] 1740485016.071416 mapwrite 0x6b5676 thru 0x6bb0e0 (0x5a6b bytes) 000049[1] 1740485016.130403 write 0x61018d thru 0x61503d (0x4eb1 bytes) 000051[0] 1740485016.154684 read 0x3cfcf6 thru 0x3d6727 (0x6a32 bytes) 000054[1] 1740485016.180692 mapwrite 0x8f937a thru 0x901e67 (0x8aee bytes) 000056[0] 1740485016.248733 mapread 0x10388f thru 0x11095b (0xd0cd bytes) 000058[0] 1740485016.277329 mapwrite 0x25a2ed thru 0x267ce4 (0xd9f8 bytes) 000059[1] 1740485016.326321 mapwrite 0x1175e4 thru 0x1273f3 (0xfe10 bytes) 000062[1] 1740485016.397529 read 0x25066a thru 0x25c973 (0xc30a bytes) 000063[1] 1740485016.404005 read 0x3e0d7d thru 0x3ece25 (0xc0a9 bytes) 000064[0] 1740485016.419843 mapwrite 0x732f11 thru 0x73bd16 (0x8e06 bytes) 000065[1] 1740485016.497771 mapwrite 0x65057c thru 0x652a5c (0x24e1 bytes) 000067[1] 1740485016.560696 read 0x33bb38 thru 0x3431f3 (0x76bc bytes) 000069[1] 1740485016.574096 read 0x27bc33 thru 0x28a885 (0xec53 bytes) 000071[0] 1740485016.577989 write 0x9fda88 thru 0x9fdc22 (0x19b bytes) 000076[0] 1740485016.613563 read 0x5ba7ed thru 0x5c3bb5 (0x93c9 bytes) 000077[1] 1740485016.629020 read 0x08d162 thru 0x09a0db (0xcf7a bytes) 000080[1] 1740485016.646886 write 0x1c1f22 thru 0x1c7b0e (0x5bed bytes) 000081[0] 1740485016.661629 write 0x29a45e thru 0x2a0149 (0x5cec bytes) 000083[0] 1740485016.697684 mapread 0x86817c thru 0x86b42a (0x32af bytes) 000084[1] 1740485016.715987 read 0x5290a4 thru 0x536fcb (0xdf28 bytes) 000089[0] 1740485016.741089 mapread 0x27a3fc thru 0x283745 (0x934a bytes) 000092[1] 1740485016.763843 mapread 0x57ad9d thru 0x57e875 (0x3ad9 bytes) 000093[0] 1740485016.784305 read 0x0ceeff thru 0x0d623d (0x733f bytes) 000094[0] 1740485016.790243 write 0x757cbe thru 0x757f45 (0x288 bytes) 000095[1] 1740485016.800257 mapread 0x42b59c thru 0x42ef67 (0x39cc bytes) 000099[0] 1740485016.836751 write 0x3bef0e thru 0x3cb4ed (0xc5e0 bytes) 000100[1] 1740485016.840774 write 0x66951a thru 0x6773fb (0xdee2 bytes) 000104[1] 1740485016.868222 write 0x0fbb9d thru 0x10b2a5 (0xf709 bytes) 000107[0] 1740485016.914868 trunc from 0x9fdc23 to 0x6eacb3 (0x312f6f bytes) 000109[0] 1740485016.964826 write 0x6857ea thru 0x685f7f (0x796 bytes) 000110[1] 1740485016.976121 mapread 0x07be36 thru 0x07fadb (0x3ca6 bytes) 000111[0] 1740485017.009698 mapwrite 0x66038f thru 0x670203 (0xfe75 bytes) 000113[0] 1740485017.081549 mapread 0x6bb615 thru 0x6c7334 (0xbd20 bytes) 000115[1] 1740485017.093757 trunc from 0x6eacb4 to 0x0322f9 (0x6b89ba bytes) 000117[1] 1740485017.129938 read 0x02d6e1 thru 0x02e6d4 (0xff4 bytes) 000118[1] 1740485017.135353 mapread 0x01c879 thru 0x02824b (0xb9d3 bytes) 000119[1] 1740485017.135862 read 0x017aad thru 0x01ef38 (0x748c bytes) 000122[0] 1740485017.137198 read 0x02428f thru 0x031303 (0xd075 bytes) 000136[0] 1740485017.220183 write 0x9be87c thru 0x9c94b9 (0xac3e bytes) 000137[1] 1740485017.235966 read 0x1e8cd5 thru 0x1f0842 (0x7b6e bytes) 000139[1] 1740485017.260594 read 0x8b9bb7 thru 0x8bbc07 (0x2051 bytes) 000140[0] 1740485017.284179 mapread 0x86e3fd thru 0x87841a (0xa01e bytes) 000142[0] 1740485017.310860 trunc from 0x9c94ba to 0x0bc58b (0x90cf2e bytes) 000144[0] 1740485017.340175 mapread 0x0b1229 thru 0x0b7ec3 (0x6c9b bytes) 000145[0] 1740485017.364181 mapread 0x00ea5e thru 0x01c302 (0xd8a5 bytes) 000146[1] 1740485017.372694 read 0x07b822 thru 0x088595 (0xcd74 bytes) 000148[0] 1740485017.421791 write 0x01388d thru 0x016284 (0x29f8 bytes) 000151[0] 1740485017.442521 trunc from 0x0bc58c to 0x3bdca7 (0x30171c bytes) 000154[0] 1740485017.475417 mapwrite 0x8ec62b thru 0x8f45dd (0x7fb3 bytes) 000156[1] 1740485017.553878 mapwrite 0x866c6d thru 0x86f5ae (0x8942 bytes) 000157[0] 1740485017.649170 mapread 0x3ad141 thru 0x3b85ea (0xb4aa bytes) 000158[0] 1740485017.684232 write 0x3de6bf thru 0x3ec726 (0xe068 bytes) truncating to largest ever: 0x87449b 000159[0] 1740485017.703260 trunc from 0x8f45de to 0x87449a (0x80143 bytes) 000163[1] 1740485017.729764 write 0x7e19d3 thru 0x7e52b3 (0x38e1 bytes) 000165[1] 1740485017.797975 mapread 0x02a55b thru 0x02be26 (0x18cc bytes) 000166[0] 1740485017.815969 read 0x557ac3 thru 0x557bf7 (0x135 bytes) 000167[1] 1740485017.835665 write 0x5f59e0 thru 0x5faf33 (0x5554 bytes) 000168[1] 1740485017.847005 trunc from 0x87449b to 0x6a9909 (0x1cab91 bytes) 000169[0] 1740485017.860916 mapread 0x33d3db thru 0x34556e (0x8194 bytes) 000171[1] 1740485017.892825 read 0x4e786d thru 0x4ec0d1 (0x4865 bytes) 000172[1] 1740485017.897413 trunc from 0x6a990a to 0x490ef0 (0x218a19 bytes) 000174[0] 1740485017.931543 write 0x6b885b thru 0x6bef85 (0x672b bytes) 000177[1] 1740485017.961064 mapread 0x1e3eab thru 0x1f2fd5 (0xf12b bytes) 000179[0] 1740485017.990321 mapread 0x2c2864 thru 0x2cddfd (0xb59a bytes) 000181[1] 1740485018.020335 read 0x15962e thru 0x15ec6b (0x563e bytes) 000182[1] 1740485018.024958 write 0x8fea92 thru 0x904fd9 (0x6548 bytes) 000184[1] 1740485018.049781 read 0x807917 thru 0x8112c7 (0x99b1 bytes) 000185[0] 1740485018.059544 write 0x234ab8 thru 0x23968d (0x4bd6 bytes) 000191[0] 1740485018.101949 write 0x144655 thru 0x14e57b (0x9f27 bytes) 000192[1] 1740485018.116147 mapread 0x8bcaf2 thru 0x8bf48c (0x299b bytes) 000193[0] 1740485018.140423 read 0x23a21e thru 0x241a5c (0x783f bytes) 000195[0] 1740485018.154270 mapread 0x37e8be thru 0x389255 (0xa998 bytes) truncating to largest ever: 0x928f1f 000196[0] 1740485018.173255 trunc from 0x904fda to 0x928f1e (0x23f45 bytes) 000197[0] 1740485018.193494 read 0x56e04c thru 0x577f94 (0x9f49 bytes) 000198[1] 1740485018.202249 mapread 0x39dc8f thru 0x3a3268 (0x55da bytes) 000201[0] 1740485018.230929 mapread 0x710a10 thru 0x71ca98 (0xc089 bytes) 000202[0] 1740485018.246679 read 0x7621b8 thru 0x770c8b (0xead4 bytes) 000205[1] 1740485018.268753 write 0x1350ea thru 0x13c80d (0x7724 bytes) 000207[0] 1740485018.283713 trunc from 0x928f1f to 0x02c570 (0x8fc9ae bytes) 000212[1] 1740485018.312346 mapread 0x026cc7 thru 0x02c570 (0x58aa bytes) 000214[1] 1740485018.322817 write 0x61a7b0 thru 0x61b823 (0x1074 bytes) 000215[0] 1740485018.331122 write 0x1d275e thru 0x1dda07 (0xb2aa bytes) 000219[0] 1740485018.352386 read 0x3ce637 thru 0x3d93db (0xada5 bytes) 000221[1] 1740485018.381845 trunc from 0x61b824 to 0x2dd983 (0x33dea0 bytes) 000224[0] 1740485018.402798 write 0x82945b thru 0x838539 (0xf0df bytes) 000228[1] 1740485018.436151 read 0x6623bb thru 0x6720a7 (0xfced bytes) 000231[1] 1740485018.484929 read 0x3980d4 thru 0x3a6cb9 (0xebe6 bytes) 000232[0] 1740485018.503749 write 0x5af5b2 thru 0x5b7dab (0x87fa bytes) 000234[0] 1740485018.515706 write 0x924844 thru 0x92b5f0 (0x6dad bytes) 000235[1] 1740485018.519848 mapread 0x06aa1b thru 0x076a41 (0xc027 bytes) 000236[1] 1740485018.544090 mapread 0x3b5ecc thru 0x3b97e5 (0x391a bytes) 000238[0] 1740485018.560397 trunc from 0x92b5f1 to 0x562d5c (0x3c8894 bytes) 000239[1] 1740485018.574216 trunc from 0x562d5d to 0x625cb9 (0xc2f5d bytes) 000245[1] 1740485018.593355 mapread 0x1adbe7 thru 0x1b276f (0x4b89 bytes) 000249[1] 1740485018.617559 write 0x2e2a6b thru 0x2e6d6e (0x4304 bytes) 000251[1] 1740485018.623933 mapwrite 0x30118f thru 0x3095ef (0x8461 bytes) 000252[0] 1740485018.663019 trunc from 0x625cba to 0x8b99ff (0x293d46 bytes) 000253[0] 1740485018.679479 mapread 0x04ab05 thru 0x05226a (0x7766 bytes) 000254[1] 1740485018.707129 trunc from 0x8b9a00 to 0x57c7a7 (0x33d258 bytes) 000255[0] 1740485018.731057 read 0x018b7f thru 0x021a63 (0x8ee5 bytes) 000257[0] 1740485018.744579 mapwrite 0x207a70 thru 0x20d36e (0x58ff bytes) 000258[1] 1740485018.785395 mapwrite 0x46510a thru 0x468bb5 (0x3aac bytes) 000264[0] 1740485018.841273 read 0x473b3d thru 0x47fb9c (0xc060 bytes) 000266[1] 1740485018.862600 mapread 0x2e7742 thru 0x2f7225 (0xfae4 bytes) 000267[0] 1740485018.877249 trunc from 0x57c7a8 to 0x458d3a (0x123a6d bytes) 000268[1] 1740485018.893379 trunc from 0x458d3b to 0x24503e (0x213cfc bytes) 000270[0] 1740485018.912319 trunc from 0x24503f to 0x161abf (0xe357f bytes) 000272[0] 1740485018.940129 read 0x0b6416 thru 0x0bedd7 (0x89c2 bytes) 000274[0] 1740485018.961753 mapwrite 0x0489a6 thru 0x04910f (0x76a bytes) 000277[0] 1740485018.984757 read 0x0b7978 thru 0x0bae27 (0x34b0 bytes) 000278[0] 1740485018.985307 write 0x7c671c thru 0x7d21c1 (0xbaa6 bytes) 000280[0] 1740485018.992841 read 0x3936f4 thru 0x3939d7 (0x2e4 bytes) 000284[0] 1740485018.998559 mapwrite 0x115d31 thru 0x116ba9 (0xe79 bytes) 000286[1] 1740485019.023102 trunc from 0x7d21c2 to 0x13c7a4 (0x695a1d bytes) 000288[1] 1740485019.042319 write 0x5031a1 thru 0x50475b (0x15bb bytes) 000292[0] 1740485019.051925 mapwrite 0x3ab8b7 thru 0x3ad852 (0x1f9c bytes) 000293[0] 1740485019.115594 read 0x057ebb thru 0x067bd3 (0xfd19 bytes) 000296[0] 1740485019.137660 read 0x3b18bf thru 0x3b852c (0x6c6e bytes) 000300[0] 1740485019.168561 trunc from 0x50475c to 0x19217a (0x3725e1 bytes) 000303[1] 1740485019.179564 mapwrite 0x1c6df2 thru 0x1c79f7 (0xc06 bytes) 000304[1] 1740485019.228037 mapwrite 0x8d8f5f thru 0x8e145b (0x84fd bytes) 000305[0] 1740485019.268945 mapwrite 0x55609e thru 0x5602dd (0xa240 bytes) 000307[1] 1740485019.314957 read 0x487635 thru 0x4910d5 (0x9aa1 bytes) 000309[0] 1740485019.323522 trunc from 0x8e145c to 0x6be05b (0x223400 bytes) 000310[1] 1740485019.341894 read 0x44ef60 thru 0x450a51 (0x1af2 bytes) 000316[1] 1740485019.363015 mapread 0x59ec77 thru 0x5a2d4d (0x40d7 bytes) 000317[0] 1740485019.372442 trunc from 0x6be05c to 0x877ab9 (0x1b9a5e bytes) 000318[1] 1740485019.387625 read 0x068415 thru 0x0743be (0xbfaa bytes) 000321[1] 1740485019.403101 trunc from 0x877aba to 0x287ec0 (0x5efbf9 bytes) 000323[1] 1740485019.422717 mapread 0x25886d thru 0x260d7a (0x850e bytes) 000325[0] 1740485019.429281 mapread 0x0e38e6 thru 0x0ed900 (0xa01b bytes) 000326[1] 1740485019.444889 mapread 0x1c22ba thru 0x1cc9d3 (0xa71a bytes) 000328[1] 1740485019.452006 mapwrite 0x1e006d thru 0x1ef1c7 (0xf15b bytes) 000332[1] 1740485019.499637 read 0x282907 thru 0x286ce8 (0x43e2 bytes) 000333[1] 1740485019.520175 mapread 0x0fd0f8 thru 0x0fecf7 (0x1c00 bytes) 000336[1] 1740485019.527090 write 0x501cac thru 0x508e41 (0x7196 bytes) 000337[1] 1740485019.534264 read 0x24101a thru 0x24d59c (0xc583 bytes) 000338[1] 1740485019.538555 write 0x35046c thru 0x35c997 (0xc52c bytes) 000346[1] 1740485019.565471 read 0x3d8522 thru 0x3e8097 (0xfb76 bytes) 000350[1] 1740485019.594142 trunc from 0x508e42 to 0x0083f7 (0x500a4a bytes) 000353[1] 1740485019.609037 read 0x0032ef thru 0x0083f7 (0x5109 bytes) 000356[1] 1740485019.623238 trunc from 0x0083f8 to 0x16f095 (0x166c9e bytes) 000357[1] 1740485019.628629 mapwrite 0x1bc6d8 thru 0x1c242e (0x5d57 bytes) 000359[1] 1740485019.675930 mapread 0x090a8a thru 0x091242 (0x7b9 bytes) 000364[0] 1740485019.689347 trunc from 0x1c242f to 0x1290b6 (0x99378 bytes) 000365[0] 1740485019.708580 trunc from 0x1290b7 to 0x0e3975 (0x45741 bytes) 000369[1] 1740485019.724279 trunc from 0x0e3976 to 0x0899e3 (0x59f92 bytes) 000370[1] 1740485019.745804 read 0x06ea75 thru 0x07bbb4 (0xd140 bytes) 000371[0] 1740485019.755342 mapwrite 0x2d6c0f thru 0x2e5366 (0xe758 bytes) 000372[0] 1740485019.793687 mapwrite 0x189cbb thru 0x19175f (0x7aa5 bytes) 000375[0] 1740485019.833098 mapread 0x292c84 thru 0x2a1842 (0xebbf bytes) 000376[1] 1740485019.843432 mapwrite 0x686df4 thru 0x68a377 (0x3584 bytes) 000377[0] 1740485019.912699 read 0x582df6 thru 0x590ef2 (0xe0fd bytes) 000378[0] 1740485019.925773 write 0x3dba87 thru 0x3dfaa2 (0x401c bytes) 000379[0] 1740485019.936514 read 0x53d2c9 thru 0x541559 (0x4291 bytes) 000380[1] 1740485019.940254 write 0x575514 thru 0x57d8fa (0x83e7 bytes) 000382[0] 1740485019.954277 read 0x043bf2 thru 0x043f21 (0x330 bytes) 000387[0] 1740485019.978233 read 0x42b83c thru 0x436d6f (0xb534 bytes) 000389[1] 1740485019.997232 read 0x205960 thru 0x20af7f (0x5620 bytes) 000391[1] 1740485020.006461 trunc from 0x68a378 to 0x2b8f2e (0x3d1449 bytes) 000392[1] 1740485020.025407 mapwrite 0x47dbbf thru 0x486570 (0x89b2 bytes) 000393[1] 1740485020.062260 trunc from 0x486571 to 0x6b5c77 (0x22f707 bytes) 000394[0] 1740485020.069661 mapwrite 0x1696ee thru 0x172068 (0x897b bytes) 000395[1] 1740485020.113769 read 0x4f433c thru 0x4f46d0 (0x395 bytes) 000396[0] 1740485020.119863 read 0x3006c1 thru 0x30f682 (0xefc2 bytes) 000397[1] 1740485020.127211 write 0x4b6535 thru 0x4bea17 (0x84e3 bytes) 000398[0] 1740485020.135287 mapread 0x2870af thru 0x28c87f (0x57d1 bytes) 000401[0] 1740485020.157837 read 0x579372 thru 0x57ecac (0x593b bytes) 000402[1] 1740485020.171427 mapread 0x6243f1 thru 0x62df03 (0x9b13 bytes) 000403[0] 1740485020.187109 read 0x19d5dd thru 0x1a7168 (0x9b8c bytes) 000405[0] 1740485020.198323 trunc from 0x6b5c78 to 0x2a8196 (0x40dae1 bytes) 000407[0] 1740485020.220271 read 0x17cc67 thru 0x18994c (0xcce6 bytes) 000408[1] 1740485020.240741 write 0x24f1fb thru 0x25c197 (0xcf9d bytes) 000411[0] 1740485020.262756 read 0x14bfc8 thru 0x156f90 (0xafc9 bytes) 000412[0] 1740485020.293819 read 0x196018 thru 0x19849c (0x2485 bytes) 000413[0] 1740485020.299855 read 0x09b789 thru 0x09bf90 (0x808 bytes) 000414[1] 1740485020.304240 write 0x332e31 thru 0x33df52 (0xb122 bytes) 000415[1] 1740485020.317324 trunc from 0x33df53 to 0x72ba7b (0x3edb29 bytes) 000417[1] 1740485020.332604 mapwrite 0x230251 thru 0x232851 (0x2601 bytes) 000418[0] 1740485020.375687 write 0x4723be thru 0x4755ef (0x3232 bytes) 000420[0] 1740485020.395008 read 0x3a6ce2 thru 0x3abf9f (0x52be bytes) 000421[0] 1740485020.402216 write 0x98237a thru 0x98e332 (0xbfb9 bytes) 000423[0] 1740485020.422272 read 0x967078 thru 0x971da4 (0xad2d bytes) 000425[1] 1740485020.435538 read 0x8634b5 thru 0x8693d3 (0x5f1f bytes) 000426[1] 1740485020.452814 mapwrite 0x17dd01 thru 0x1852c6 (0x75c6 bytes) 000427[0] 1740485020.500570 mapread 0x84b73c thru 0x85aed4 (0xf799 bytes) 000429[0] 1740485020.527837 mapwrite 0x4cee26 thru 0x4d72aa (0x8485 bytes) 000434[0] 1740485020.571734 write 0x24009e thru 0x24abd5 (0xab38 bytes) 000435[0] 1740485020.578295 write 0x1a3105 thru 0x1b2f48 (0xfe44 bytes) 000438[1] 1740485020.592330 write 0x45df2e thru 0x46663b (0x870e bytes) 000440[0] 1740485020.610077 read 0x7da181 thru 0x7e0871 (0x66f1 bytes) 000441[1] 1740485020.638558 trunc from 0x98e333 to 0x2c905c (0x6c52d6 bytes) 000442[1] 1740485020.667602 mapread 0x0a23fa thru 0x0ad3ad (0xafb4 bytes) 000445[1] 1740485020.698282 read 0x03a3d7 thru 0x03e1dd (0x3e07 bytes) 000448[0] 1740485020.734791 mapread 0x2b2265 thru 0x2b8ce7 (0x6a83 bytes) 000449[0] 1740485020.767195 write 0x8e7804 thru 0x8f2130 (0xa92d bytes) 000450[1] 1740485020.776629 mapread 0x0cd389 thru 0x0d82c6 (0xaf3e bytes) 000457[0] 1740485020.831442 write 0x919c07 thru 0x91b86a (0x1c64 bytes) 000460[0] 1740485020.850746 write 0x168576 thru 0x16ce8e (0x4919 bytes) 000469[0] 1740485020.884058 write 0x7ed788 thru 0x7f9e63 (0xc6dc bytes) 000470[1] 1740485020.894970 read 0x0f220b thru 0x0f8745 (0x653b bytes) 000473[0] 1740485020.904318 write 0x4445f8 thru 0x44aa37 (0x6440 bytes) 000474[0] 1740485020.912246 trunc from 0x91b86b to 0x06b044 (0x8b0826 bytes) 000475[1] 1740485020.923948 read 0x05fe78 thru 0x064632 (0x47bb bytes) 000477[1] 1740485020.941354 trunc from 0x06b045 to 0x6b8b5d (0x64db19 bytes) 000480[1] 1740485020.970084 mapread 0x19d5da thru 0x1a2a47 (0x546e bytes) 000481[1] 1740485020.982535 mapread 0x03012e thru 0x03bd25 (0xbbf8 bytes) 000486[1] 1740485020.998555 trunc from 0x6b8b5e to 0x868d4a (0x1b01ed bytes) 000489[0] 1740485021.015925 write 0x03e833 thru 0x0461bc (0x798a bytes) 000490[1] 1740485021.028266 read 0x5feb9b thru 0x60d1fe (0xe664 bytes) 000492[1] 1740485021.046448 trunc from 0x868d4b to 0x1cf5dc (0x69976e bytes) 000493[1] 1740485021.059732 mapwrite 0x2744e0 thru 0x283317 (0xee38 bytes) 000497[0] 1740485021.128673 mapwrite 0x24af9f thru 0x253cde (0x8d40 bytes) 000498[1] 1740485021.172339 mapwrite 0x3943e7 thru 0x3978bc (0x34d6 bytes) 000499[0] 1740485021.217659 mapread 0x09ab07 thru 0x0a7a69 (0xcf63 bytes) 000500[0] 1740485021.238619 trunc from 0x3978bd to 0x1921d4 (0x2056e8 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5021 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000003[0] 1740485021.533813 mapwrite 0x841000 thru 0x8476dc (0x66dd bytes) 000004[0] 1740485021.583659 mapread 0x385000 thru 0x3932e3 (0xe2e4 bytes) 000009[0] 1740485021.605571 mapread 0x6c5000 thru 0x6ce5b1 (0x95b2 bytes) 000010[0] 1740485021.615308 write 0x0ad000 thru 0x0bafa7 (0xdfa8 bytes) 000011[1] 1740485021.618346 mapwrite 0x16d000 thru 0x1703ad (0x33ae bytes) 000012[0] 1740485021.652958 mapread 0x5bd000 thru 0x5bde75 (0xe76 bytes) 000014[0] 1740485021.661781 mapwrite 0x60e000 thru 0x6179f5 (0x99f6 bytes) truncating to largest ever: 0x33f1a2 000015[0] 1740485021.687474 trunc from 0x8476dd to 0x33f1a1 (0x50853b bytes) 000016[0] 1740485021.697879 mapread 0x1b3000 thru 0x1b43da (0x13db bytes) 000017[1] 1740485021.705648 mapwrite 0x5c1000 thru 0x5c46fa (0x36fb bytes) 000019[0] 1740485021.740620 write 0x98b000 thru 0x997cc5 (0xccc6 bytes) 000025[0] 1740485021.763266 write 0x886000 thru 0x895318 (0xf319 bytes) 000026[0] 1740485021.768937 mapwrite 0x6b5000 thru 0x6b6be6 (0x1be7 bytes) 000028[0] 1740485021.788973 mapread 0x32e000 thru 0x33c706 (0xe707 bytes) truncating to largest ever: 0x4a7ff5 000031[0] 1740485021.798448 trunc from 0x997cc6 to 0x4a7ff4 (0x4efcd1 bytes) 000037[0] 1740485021.817189 write 0x9f9000 thru 0x9ff336 (0x6337 bytes) 000039[1] 1740485021.822266 mapread 0x77f000 thru 0x7863af (0x73b0 bytes) 000041[0] 1740485021.834210 mapwrite 0x237000 thru 0x244d0c (0xdd0d bytes) 000042[0] 1740485021.877228 mapwrite 0x942000 thru 0x94a115 (0x8116 bytes) 000044[0] 1740485021.901646 trunc from 0x9ff337 to 0x19f2cb (0x86006b bytes) 000045[0] 1740485021.908000 mapwrite 0x055000 thru 0x05e0f0 (0x90f1 bytes) 000047[1] 1740485021.942245 mapwrite 0x279000 thru 0x287dec (0xeded bytes) 000048[0] 1740485021.975823 mapwrite 0x581000 thru 0x589c54 (0x8c55 bytes) 000049[1] 1740485022.005377 mapwrite 0x972000 thru 0x981979 (0xf97a bytes) 000055[0] 1740485022.044711 mapwrite 0x0a1000 thru 0x0a4bb5 (0x3bb6 bytes) 000056[1] 1740485022.065978 trunc from 0x98197a to 0x21bb48 (0x765e31 bytes) 000057[0] 1740485022.077361 write 0x92f000 thru 0x930af4 (0x1af5 bytes) 000058[0] 1740485022.086333 write 0x846000 thru 0x84b931 (0x5932 bytes) 000059[0] 1740485022.090789 mapwrite 0x388000 thru 0x395bc4 (0xdbc5 bytes) 000062[1] 1740485022.122169 mapwrite 0x45c000 thru 0x45c635 (0x636 bytes) 000067[0] 1740485022.149025 mapread 0x84d000 thru 0x84f50c (0x250d bytes) 000068[1] 1740485022.160447 mapread 0x13c000 thru 0x13c3d2 (0x3d3 bytes) 000072[0] 1740485022.162215 write 0x33c000 thru 0x3437a7 (0x77a8 bytes) 000074[1] 1740485022.170584 trunc from 0x930af5 to 0x16c018 (0x7c4adc bytes) truncating to largest ever: 0x726e14 000075[1] 1740485022.180312 trunc from 0x16c019 to 0x726e13 (0x5badfb bytes) 000076[1] 1740485022.185173 read_OD 0x210000 thru 0x216fff (0x7000 bytes) 000077[1] 1740485022.189439 trunc from 0x726e14 to 0x5da31a (0x14caf9 bytes) 000080[0] 1740485022.200625 write 0x33d000 thru 0x3429f4 (0x59f5 bytes) 000081[1] 1740485022.205176 mapwrite 0x763000 thru 0x763e97 (0xe98 bytes) 000083[0] 1740485022.238345 read 0x31d000 thru 0x3238c7 (0x68c8 bytes) 000084[0] 1740485022.244355 read 0x11e000 thru 0x12688a (0x888b bytes) truncating to largest ever: 0x89e7a7 000086[1] 1740485022.247043 trunc from 0x763e98 to 0x89e7a6 (0x13a90f bytes) 000089[1] 1740485022.268695 read_OD 0x10d000 thru 0x10dfff (0x1000 bytes) 000090[1] 1740485022.272263 read_OD 0x812000 thru 0x81afff (0x9000 bytes) 000092[1] 1740485022.276996 read_OD 0x291000 thru 0x297fff (0x7000 bytes) 000093[0] 1740485022.278934 read 0x6aa000 thru 0x6b9b39 (0xfb3a bytes) 000094[1] 1740485022.281857 mapwrite 0x66a000 thru 0x66a619 (0x61a bytes) 000096[1] 1740485022.300224 mapread 0x79c000 thru 0x7a1fe2 (0x5fe3 bytes) 000097[1] 1740485022.304693 trunc from 0x89e7a7 to 0x80673d (0x98069 bytes) 000100[0] 1740485022.309786 read 0x111000 thru 0x1189b4 (0x79b5 bytes) 000101[0] 1740485022.314859 mapread 0x7c1000 thru 0x7cc462 (0xb463 bytes) 000103[0] 1740485022.322864 mapread 0x69c000 thru 0x69d5a2 (0x15a3 bytes) 000104[0] 1740485022.325602 write 0x566000 thru 0x56fa6f (0x9a70 bytes) 000106[0] 1740485022.338927 read 0x1c9000 thru 0x1d78ab (0xe8ac bytes) 000107[1] 1740485022.341390 read_OD 0x031000 thru 0x032fff (0x2000 bytes) 000109[0] 1740485022.351099 read 0x4e9000 thru 0x4ecdbc (0x3dbd bytes) 000110[1] 1740485022.354137 trunc from 0x80673e to 0x2542c0 (0x5b247d bytes) 000114[0] 1740485022.375343 trunc from 0x2542c1 to 0x5736d9 (0x31f419 bytes) 000115[1] 1740485022.386226 read_OD 0x435000 thru 0x436fff (0x2000 bytes) 000117[1] 1740485022.395001 write_OD 0x0f5000 thru 0x100fff (0xc000 bytes) 000120[1] 1740485022.414481 write_OD 0x474000 thru 0x478fff (0x5000 bytes) 000121[1] 1740485022.432679 mapread 0x1c0000 thru 0x1cb938 (0xb939 bytes) 000124[0] 1740485022.468245 write 0x324000 thru 0x326146 (0x2147 bytes) 000127[0] 1740485022.490844 read 0x167000 thru 0x168c11 (0x1c12 bytes) 000129[1] 1740485022.498102 write_OD 0x462000 thru 0x462fff (0x1000 bytes) 000137[0] 1740485022.523829 read 0x06c000 thru 0x06de9d (0x1e9e bytes) 000139[0] 1740485022.526258 write 0x26d000 thru 0x26ff1e (0x2f1f bytes) 000141[0] 1740485022.538893 mapwrite 0x775000 thru 0x77d4a2 (0x84a3 bytes) 000144[0] 1740485022.575454 trunc from 0x77d4a3 to 0x0ec51a (0x690f88 bytes) 000145[1] 1740485022.580689 read_OD 0x023000 thru 0x024fff (0x2000 bytes) 000148[1] 1740485022.590299 trunc from 0x0ec51b to 0x84d709 (0x7611ef bytes) 000150[0] 1740485022.604823 mapwrite 0x71e000 thru 0x724c15 (0x6c16 bytes) 000151[0] 1740485022.632312 mapread 0x73b000 thru 0x73efb0 (0x3fb1 bytes) 000152[1] 1740485022.634768 trunc from 0x84d70a to 0x86edb7 (0x216ae bytes) 000155[1] 1740485022.646945 trunc from 0x86edb8 to 0x195a5f (0x6d9358 bytes) 000157[1] 1740485022.656893 read_OD 0x0ae000 thru 0x0b7fff (0xa000 bytes) 000158[0] 1740485022.661714 write 0x7a7000 thru 0x7b6261 (0xf262 bytes) 000159[0] 1740485022.669290 write 0x37e000 thru 0x37e52c (0x52d bytes) 000160[1] 1740485022.672307 write_OD 0x3fe000 thru 0x402fff (0x5000 bytes) 000163[1] 1740485022.700757 mapread 0x10f000 thru 0x117590 (0x8591 bytes) 000165[0] 1740485022.737192 mapwrite 0x5b3000 thru 0x5be5d5 (0xb5d6 bytes) 000166[1] 1740485022.771828 trunc from 0x7b6262 to 0x3013e9 (0x4b4e78 bytes) 000168[1] 1740485022.783794 trunc from 0x3013ea to 0x533dea (0x232a01 bytes) 000170[1] 1740485022.794272 write_OD 0x87e000 thru 0x87ffff (0x2000 bytes) 000171[1] 1740485022.813892 write_OD 0x1a6000 thru 0x1b2fff (0xd000 bytes) truncating to largest ever: 0x9135d0 000172[1] 1740485022.834773 trunc from 0x880000 to 0x9135cf (0x935d0 bytes) 000173[0] 1740485022.842464 mapread 0x45d000 thru 0x467ed2 (0xaed3 bytes) 000179[1] 1740485022.857239 trunc from 0x9135d0 to 0x8c9245 (0x4a38a bytes) 000180[1] 1740485022.865245 read_OD 0x4b8000 thru 0x4bafff (0x3000 bytes) 000182[0] 1740485022.869558 mapread 0x33f000 thru 0x348523 (0x9524 bytes) 000184[0] 1740485022.875289 mapread 0x89b000 thru 0x8a5301 (0xa302 bytes) 000187[0] 1740485022.883752 mapread 0x272000 thru 0x27b5c8 (0x95c9 bytes) 000188[0] 1740485022.889313 trunc from 0x8c9246 to 0x178fe7 (0x75025e bytes) 000189[0] 1740485022.901525 trunc from 0x178fe8 to 0x41d313 (0x2a432c bytes) 000190[0] 1740485022.910047 trunc from 0x41d314 to 0x559d11 (0x13c9fe bytes) 000193[0] 1740485022.925276 mapwrite 0x2b0000 thru 0x2b1847 (0x1848 bytes) 000194[0] 1740485022.956892 mapwrite 0x530000 thru 0x535bff (0x5c00 bytes) 000195[1] 1740485022.982107 mapwrite 0x5ab000 thru 0x5b7cc9 (0xccca bytes) 000198[1] 1740485023.032223 trunc from 0x5b7cca to 0x8e573a (0x32da71 bytes) 000199[0] 1740485023.037584 read 0x34b000 thru 0x34ec0d (0x3c0e bytes) 000203[1] 1740485023.045774 mapwrite 0x83f000 thru 0x84e15d (0xf15e bytes) 000206[0] 1740485023.087322 mapwrite 0x238000 thru 0x23801a (0x1b bytes) 000209[1] 1740485023.130268 trunc from 0x8e573b to 0x62b9e7 (0x2b9d53 bytes) 000210[1] 1740485023.142932 write_OD 0x133000 thru 0x134fff (0x2000 bytes) 000215[0] 1740485023.167388 write 0x2d5000 thru 0x2d9e68 (0x4e69 bytes) 000216[1] 1740485023.172307 trunc from 0x62b9e8 to 0x38eaa3 (0x29cf44 bytes) 000218[1] 1740485023.193592 trunc from 0x38eaa4 to 0x1da20f (0x1b4894 bytes) 000219[1] 1740485023.207588 write_OD 0x5ad000 thru 0x5adfff (0x1000 bytes) 000220[1] 1740485023.234768 write_OD 0x140000 thru 0x14cfff (0xd000 bytes) 000221[1] 1740485023.254382 mapwrite 0x387000 thru 0x38c2b0 (0x52b1 bytes) 000222[0] 1740485023.290170 write 0x7b7000 thru 0x7c59d0 (0xe9d1 bytes) 000223[0] 1740485023.294655 write 0x45d000 thru 0x461e04 (0x4e05 bytes) 000224[1] 1740485023.300847 mapread 0x76e000 thru 0x76f805 (0x1806 bytes) 000225[0] 1740485023.312711 mapread 0x10b000 thru 0x110f2f (0x5f30 bytes) 000226[0] 1740485023.320688 mapwrite 0x3ff000 thru 0x40acc0 (0xbcc1 bytes) 000229[0] 1740485023.360545 read 0x38b000 thru 0x38d225 (0x2226 bytes) 000230[1] 1740485023.362101 mapwrite 0x6f6000 thru 0x6fe252 (0x8253 bytes) 000233[1] 1740485023.399258 mapwrite 0x0f5000 thru 0x0fdbca (0x8bcb bytes) 000234[1] 1740485023.428972 read_OD 0x2a2000 thru 0x2a3fff (0x2000 bytes) 000235[1] 1740485023.435580 trunc from 0x7c59d1 to 0x4b6b03 (0x30eecd bytes) 000236[0] 1740485023.443964 mapwrite 0x17a000 thru 0x17f376 (0x5377 bytes) 000237[0] 1740485023.487505 mapwrite 0x363000 thru 0x364b7d (0x1b7e bytes) 000239[1] 1740485023.525141 read_OD 0x0ae000 thru 0x0b6fff (0x9000 bytes) 000241[0] 1740485023.535726 mapread 0x187000 thru 0x18d40a (0x640b bytes) 000242[1] 1740485023.542615 trunc from 0x4b6b04 to 0x145fc6 (0x370b3d bytes) 000246[0] 1740485023.569830 mapread 0x00b000 thru 0x010c15 (0x5c16 bytes) 000254[0] 1740485023.584372 mapwrite 0x382000 thru 0x389ab0 (0x7ab1 bytes) 000255[1] 1740485023.628728 mapwrite 0x7fc000 thru 0x8007be (0x47bf bytes) 000260[0] 1740485023.684669 read 0x45f000 thru 0x46d379 (0xe37a bytes) 000263[0] 1740485023.692353 read 0x2db000 thru 0x2e97aa (0xe7ab bytes) 000268[0] 1740485023.696145 mapwrite 0x6ed000 thru 0x6f58a4 (0x88a5 bytes) 000269[1] 1740485023.728358 write_OD 0x6ea000 thru 0x6f7fff (0xe000 bytes) 000270[1] 1740485023.754142 mapread 0x122000 thru 0x126d38 (0x4d39 bytes) 000273[1] 1740485023.762618 read_OD 0x13c000 thru 0x141fff (0x6000 bytes) 000274[1] 1740485023.765858 mapwrite 0x17f000 thru 0x187b9c (0x8b9d bytes) 000275[1] 1740485023.796745 mapread 0x369000 thru 0x3772da (0xe2db bytes) 000277[1] 1740485023.812425 trunc from 0x8007bf to 0x6605fd (0x1a01c1 bytes) 000280[0] 1740485023.828585 mapwrite 0x6b2000 thru 0x6bf18f (0xd190 bytes) 000281[0] 1740485023.868190 write 0x15b000 thru 0x16769c (0xc69d bytes) 000282[1] 1740485023.871445 trunc from 0x6bf190 to 0x0ad41d (0x611d72 bytes) 000284[1] 1740485023.890748 mapread 0x012000 thru 0x01c875 (0xa876 bytes) 000286[1] 1740485023.902561 trunc from 0x0ad41e to 0x66a904 (0x5bd4e7 bytes) 000288[1] 1740485023.915889 read_OD 0x251000 thru 0x251fff (0x1000 bytes) 000289[1] 1740485023.920420 trunc from 0x66a905 to 0x71453e (0xa9c3a bytes) 000290[0] 1740485023.933568 write 0x924000 thru 0x931cc5 (0xdcc6 bytes) 000293[0] 1740485023.950768 mapwrite 0x2c2000 thru 0x2d174c (0xf74d bytes) 000296[1] 1740485023.984489 mapread 0x161000 thru 0x1657b0 (0x47b1 bytes) 000297[0] 1740485023.999805 write 0x77b000 thru 0x783dcd (0x8dce bytes) 000300[0] 1740485024.013219 write 0x4c8000 thru 0x4cd7bc (0x57bd bytes) 000302[0] 1740485024.017029 mapwrite 0x41a000 thru 0x41b30b (0x130c bytes) 000303[1] 1740485024.044814 read_OD 0x20c000 thru 0x20dfff (0x2000 bytes) 000306[0] 1740485024.053317 read 0x192000 thru 0x1a09c2 (0xe9c3 bytes) 000313[0] 1740485024.058668 read 0x4c1000 thru 0x4c48b1 (0x38b2 bytes) 000314[0] 1740485024.060744 write 0x7d8000 thru 0x7da181 (0x2182 bytes) 000316[1] 1740485024.069931 write_OD 0x08c000 thru 0x097fff (0xc000 bytes) 000317[0] 1740485024.095226 mapwrite 0x31b000 thru 0x3294de (0xe4df bytes) 000318[0] 1740485024.135294 mapwrite 0x53a000 thru 0x53ecda (0x4cdb bytes) 000319[1] 1740485024.162446 mapwrite 0x95d000 thru 0x96356a (0x656b bytes) 000320[1] 1740485024.194610 write_OD 0x01e000 thru 0x02cfff (0xf000 bytes) 000322[1] 1740485024.215725 mapwrite 0x3f4000 thru 0x3f8583 (0x4584 bytes) 000323[1] 1740485024.254282 mapread 0x407000 thru 0x40b777 (0x4778 bytes) 000324[1] 1740485024.259928 mapwrite 0x4c3000 thru 0x4cf416 (0xc417 bytes) 000325[1] 1740485024.287341 mapwrite 0x006000 thru 0x00786c (0x186d bytes) 000327[0] 1740485024.314585 trunc from 0x96356b to 0x56e4fe (0x3f506c bytes) 000328[1] 1740485024.324472 write_OD 0x42e000 thru 0x43afff (0xd000 bytes) 000329[1] 1740485024.344207 trunc from 0x56e4ff to 0x645146 (0xd6c48 bytes) 000331[1] 1740485024.356330 trunc from 0x645147 to 0x2610f8 (0x3e404e bytes) 000332[1] 1740485024.365353 mapwrite 0x970000 thru 0x97dcb0 (0xdcb1 bytes) 000336[0] 1740485024.410102 mapread 0x4ba000 thru 0x4bfcab (0x5cac bytes) 000337[0] 1740485024.421162 mapread 0x511000 thru 0x515cac (0x4cad bytes) 000340[0] 1740485024.424812 trunc from 0x97dcb1 to 0x4b01da (0x4cdad6 bytes) 000341[1] 1740485024.436937 mapwrite 0x2b6000 thru 0x2b6f96 (0xf97 bytes) 000343[0] 1740485024.468265 write 0x7c0000 thru 0x7c5310 (0x5311 bytes) 000345[1] 1740485024.477775 read_OD 0x3b5000 thru 0x3bcfff (0x8000 bytes) 000346[1] 1740485024.487965 write_OD 0x8a7000 thru 0x8aefff (0x8000 bytes) 000348[1] 1740485024.504634 trunc from 0x8af000 to 0x621773 (0x28d88c bytes) 000349[0] 1740485024.517429 mapread 0x54d000 thru 0x5540f0 (0x70f1 bytes) 000350[0] 1740485024.525128 write 0x9bd000 thru 0x9bfe3e (0x2e3f bytes) 000351[0] 1740485024.533149 read 0x487000 thru 0x4906cf (0x96d0 bytes) 000355[0] 1740485024.549688 mapread 0x091000 thru 0x09ed49 (0xdd4a bytes) 000356[1] 1740485024.565832 write_OD 0x16f000 thru 0x170fff (0x2000 bytes) 000357[1] 1740485024.586802 mapwrite 0x630000 thru 0x6315e3 (0x15e4 bytes) 000359[0] 1740485024.630880 mapread 0x6ee000 thru 0x6f10d1 (0x30d2 bytes) 000360[0] 1740485024.642488 mapread 0x878000 thru 0x87e93b (0x693c bytes) 000361[0] 1740485024.648839 trunc from 0x9bfe3f to 0x0be6d8 (0x901766 bytes) 000363[1] 1740485024.669711 write_OD 0x0fa000 thru 0x105fff (0xc000 bytes) 000364[0] 1740485024.696309 trunc from 0x106000 to 0x55af82 (0x454f83 bytes) 000365[0] 1740485024.709371 trunc from 0x55af83 to 0x3dcc90 (0x17e2f2 bytes) 000366[1] 1740485024.721457 read_OD 0x0d6000 thru 0x0e1fff (0xc000 bytes) 000367[1] 1740485024.737539 mapwrite 0x3d4000 thru 0x3e2f1b (0xef1c bytes) 000368[0] 1740485024.806477 mapread 0x355000 thru 0x35fa41 (0xaa42 bytes) 000369[1] 1740485024.828296 write_OD 0x307000 thru 0x30dfff (0x7000 bytes) 000371[1] 1740485024.851476 trunc from 0x3e2f1c to 0x13b806 (0x2a7715 bytes) 000372[0] 1740485024.859140 mapread 0x015000 thru 0x018fad (0x3fae bytes) 000376[1] 1740485024.877928 mapwrite 0x8e3000 thru 0x8f2f5c (0xff5d bytes) 000377[1] 1740485024.969068 write_OD 0x62e000 thru 0x633fff (0x6000 bytes) 000379[0] 1740485025.000953 write 0x1d7000 thru 0x1db7f8 (0x47f9 bytes) 000380[0] 1740485025.010046 mapwrite 0x6bc000 thru 0x6c1426 (0x5427 bytes) 000381[0] 1740485025.034295 trunc from 0x8f2f5d to 0x66cfb2 (0x285faa bytes) 000382[0] 1740485025.045543 trunc from 0x66cfb3 to 0x47a238 (0x1f2d7a bytes) 000384[0] 1740485025.052808 mapread 0x36c000 thru 0x371b72 (0x5b73 bytes) 000387[0] 1740485025.068039 read 0x454000 thru 0x4548f4 (0x8f5 bytes) 000389[1] 1740485025.070949 write_OD 0x30e000 thru 0x31bfff (0xe000 bytes) 000390[0] 1740485025.091140 read 0x42f000 thru 0x431f49 (0x2f4a bytes) 000396[1] 1740485025.095581 mapread 0x34f000 thru 0x356f7e (0x7f7f bytes) 000397[1] 1740485025.107988 read_OD 0x154000 thru 0x15afff (0x7000 bytes) 000398[0] 1740485025.111668 mapread 0x16b000 thru 0x17ae0f (0xfe10 bytes) 000404[1] 1740485025.124013 mapwrite 0x3c2000 thru 0x3d04d2 (0xe4d3 bytes) 000406[0] 1740485025.168831 write 0x503000 thru 0x510fe3 (0xdfe4 bytes) 000409[0] 1740485025.173801 write 0x9ea000 thru 0x9f4bf6 (0xabf7 bytes) 000410[0] 1740485025.175514 write 0x0b1000 thru 0x0b2d87 (0x1d88 bytes) 000411[1] 1740485025.183244 trunc from 0x9f4bf7 to 0x7abb2f (0x2490c7 bytes) 000413[0] 1740485025.197837 write 0x7fa000 thru 0x7fc215 (0x2216 bytes) 000414[0] 1740485025.204347 read 0x0a2000 thru 0x0aee2c (0xce2d bytes) 000417[0] 1740485025.213997 trunc from 0x7fc216 to 0x6d3c7c (0x128599 bytes) 000419[1] 1740485025.223119 read_OD 0x699000 thru 0x6a6fff (0xe000 bytes) 000421[1] 1740485025.234650 mapwrite 0x836000 thru 0x83bb6b (0x5b6c bytes) 000423[0] 1740485025.282274 trunc from 0x83bb6c to 0x462239 (0x3d9932 bytes) 000427[1] 1740485025.307714 read_OD 0x3bf000 thru 0x3c0fff (0x2000 bytes) 000431[1] 1740485025.322747 write_OD 0x68b000 thru 0x695fff (0xb000 bytes) 000432[0] 1740485025.348178 mapread 0x39c000 thru 0x3a8c68 (0xcc69 bytes) 000436[0] 1740485025.362541 mapwrite 0x5f1000 thru 0x5f4ee5 (0x3ee6 bytes) 000439[1] 1740485025.396173 mapwrite 0x51d000 thru 0x51d2f1 (0x2f2 bytes) 000440[1] 1740485025.423044 mapwrite 0x872000 thru 0x878a03 (0x6a04 bytes) 000441[1] 1740485025.456526 mapwrite 0x332000 thru 0x33296b (0x96c bytes) truncating to largest ever: 0x9afe1b 000443[1] 1740485025.483434 trunc from 0x878a04 to 0x9afe1a (0x137417 bytes) 000446[0] 1740485025.495141 write 0x913000 thru 0x919df1 (0x6df2 bytes) 000447[0] 1740485025.499249 write 0x1f9000 thru 0x2005cc (0x75cd bytes) 000453[0] 1740485025.517210 mapwrite 0x9e3000 thru 0x9e6bcd (0x3bce bytes) 000455[1] 1740485025.545864 mapread 0x31c000 thru 0x31c962 (0x963 bytes) 000456[1] 1740485025.557300 trunc from 0x9e6bce to 0x187bee (0x85efdf bytes) 000458[1] 1740485025.565033 trunc from 0x187bef to 0x933b24 (0x7abf36 bytes) 000459[1] 1740485025.570568 write_OD 0x6ec000 thru 0x6f5fff (0xa000 bytes) 000460[1] 1740485025.590430 write_OD 0x4f2000 thru 0x4fffff (0xe000 bytes) 000463[1] 1740485025.611285 write_OD 0x9b9000 thru 0x9c7fff (0xf000 bytes) 000469[0] 1740485025.632840 write 0x50e000 thru 0x517de8 (0x9de9 bytes) 000470[1] 1740485025.638584 mapwrite 0x128000 thru 0x136965 (0xe966 bytes) 000471[1] 1740485025.673731 trunc from 0x9c8000 to 0x7e7c2a (0x1e03d5 bytes) 000475[0] 1740485025.687612 mapread 0x2e6000 thru 0x2eca88 (0x6a89 bytes) 000477[0] 1740485025.697178 mapwrite 0x2b3000 thru 0x2be84f (0xb850 bytes) 000480[0] 1740485025.731765 read 0x270000 thru 0x2793ff (0x9400 bytes) 000484[1] 1740485025.751272 mapwrite 0x091000 thru 0x0958dd (0x48de bytes) 000487[0] 1740485025.789646 write 0x236000 thru 0x243ff1 (0xdff2 bytes) 000489[0] 1740485025.804078 mapwrite 0x777000 thru 0x781ef6 (0xaef7 bytes) 000490[0] 1740485025.837925 mapread 0x76d000 thru 0x772804 (0x5805 bytes) 000492[0] 1740485025.838881 trunc from 0x7e7c2b to 0x0d80eb (0x70fb3f bytes) 000494[1] 1740485025.854859 trunc from 0x0d80ec to 0x366b57 (0x28ea6c bytes) 000498[0] 1740485025.876074 trunc from 0x366b58 to 0x4af98f (0x148e38 bytes) 000499[0] 1740485025.889259 mapread 0x20e000 thru 0x21089e (0x289f bytes) All operations completed A-OK! PASS 16a (13s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 07:03:47 (1740485027) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.159388 s, 52.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.136876 s, 61.3 MB/s Chance of close/open is 1 in 50 Seed set to 5028 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x138909 000001[0] 1740485028.328135 trunc from 00000000 to 0x138908 (0x138909 bytes) 000002[0] 1740485028.340651 read 0x0c7890 thru 0x0c90d1 (0x1842 bytes) 000004[1] 1740485028.349001 write 0x08a9db thru 0x08c193 (0x17b9 bytes) 000006[0] 1740485028.361193 mapwrite 0x3d91d8 thru 0x3d97ae (0x5d7 bytes) 000008[0] 1740485028.404412 mapwrite 0x3e3ca5 thru 0x3e3d79 (0xd5 bytes) 000009[0] 1740485028.426429 mapread 0x333afb thru 0x333cfd (0x203 bytes) truncating to largest ever: 0x65e39e 000011[0] 1740485028.438747 trunc from 0x3e3d7a to 0x65e39d (0x27a624 bytes) 000017[0] 1740485028.462319 write 0x39a24c thru 0x39a92a (0x6df bytes) 000018[0] 1740485028.465594 mapread 0x57de39 thru 0x57f5e1 (0x17a9 bytes) 000019[1] 1740485028.485260 read 0x157eea thru 0x159e25 (0x1f3c bytes) 000021[0] 1740485028.495185 mapwrite 0x62fb3a thru 0x62fb8a (0x51 bytes) 000023[1] 1740485028.542612 write 0x58c351 thru 0x58e2e0 (0x1f90 bytes) 000024[1] 1740485028.551829 read 0x0c0ace thru 0x0c0b3a (0x6d bytes) 000026[0] 1740485028.564603 trunc from 0x65e39e to 0x23a31d (0x424080 bytes) 000028[1] 1740485028.583649 read 0x2075a9 thru 0x20888f (0x12e7 bytes) 000029[1] 1740485028.611545 read 0x053cc1 thru 0x055378 (0x16b8 bytes) 000032[1] 1740485028.613876 write 0x8c2f64 thru 0x8c4e59 (0x1ef6 bytes) 000033[0] 1740485028.623834 mapread 0x8bab17 thru 0x8bb115 (0x5ff bytes) 000036[1] 1740485028.639697 mapread 0x6a4779 thru 0x6a5823 (0x10ab bytes) 000038[1] 1740485028.673971 mapread 0x373b34 thru 0x374204 (0x6d1 bytes) truncating to largest ever: 0x6e4863 000047[1] 1740485028.677303 trunc from 0x8c4e5a to 0x6e4862 (0x1e05f7 bytes) 000048[0] 1740485028.694060 read 0x223574 thru 0x225120 (0x1bad bytes) 000049[0] 1740485028.703386 write 0x64bd4e thru 0x64da24 (0x1cd7 bytes) 000050[0] 1740485028.711077 trunc from 0x6e4863 to 0x41ab58 (0x2c9d0a bytes) 000051[1] 1740485028.723181 read 0x28bc83 thru 0x28bd77 (0xf5 bytes) 000054[1] 1740485028.737097 read 0x0f202a thru 0x0f2b51 (0xb28 bytes) 000055[0] 1740485028.742955 trunc from 0x41ab59 to 0x3b4d49 (0x65e0f bytes) 000056[1] 1740485028.755848 read 0x022e14 thru 0x023f41 (0x112e bytes) 000058[1] 1740485028.791577 mapwrite 0x41a312 thru 0x41b03b (0xd2a bytes) 000059[1] 1740485028.835659 read 0x20403f thru 0x2054eb (0x14ad bytes) 000060[1] 1740485028.843892 write 0x1cdbca thru 0x1cdc9e (0xd5 bytes) 000061[0] 1740485028.849535 trunc from 0x41b03c to 0x1927fc (0x28883f bytes) 000064[0] 1740485028.882039 write 0x830970 thru 0x8312c3 (0x954 bytes) 000065[1] 1740485028.889044 mapread 0x46c5dd thru 0x46e068 (0x1a8c bytes) truncating to largest ever: 0x843c7f 000067[0] 1740485028.910671 trunc from 0x8312c4 to 0x843c7e (0x129bb bytes) 000070[1] 1740485028.925378 trunc from 0x843c7f to 0x634fbc (0x20ecc2 bytes) 000074[1] 1740485028.952856 mapread 0x5bdaf6 thru 0x5bf907 (0x1e12 bytes) 000075[0] 1740485028.962711 write 0x800d73 thru 0x8019f9 (0xc87 bytes) 000079[0] 1740485028.984531 write 0x5aa4aa thru 0x5ab714 (0x126b bytes) 000081[0] 1740485028.998765 write 0x9360d5 thru 0x937372 (0x129e bytes) 000083[0] 1740485029.002995 mapwrite 0x86de20 thru 0x86fa39 (0x1c1a bytes) 000084[1] 1740485029.050861 write 0x1c2cb7 thru 0x1c45a5 (0x18ef bytes) 000085[0] 1740485029.061276 read 0x3406b0 thru 0x341f0e (0x185f bytes) 000087[0] 1740485029.076546 mapwrite 0x59f8d5 thru 0x5a0198 (0x8c4 bytes) 000090[1] 1740485029.105628 mapwrite 0x7f7e2b thru 0x7f907f (0x1255 bytes) truncating to largest ever: 0x8e58fc 000094[1] 1740485029.147254 trunc from 0x937373 to 0x8e58fb (0x51a77 bytes) 000099[1] 1740485029.167701 read 0x324152 thru 0x3254a3 (0x1352 bytes) 000100[1] 1740485029.169695 mapwrite 0x545c64 thru 0x547788 (0x1b25 bytes) 000101[0] 1740485029.197082 mapread 0x880269 thru 0x880b87 (0x91f bytes) 000102[1] 1740485029.212546 trunc from 0x8e58fc to 0x74ab22 (0x19add9 bytes) 000103[1] 1740485029.222112 write 0x97757f thru 0x9780d9 (0xb5b bytes) 000104[0] 1740485029.223426 read 0x51d286 thru 0x51d674 (0x3ef bytes) 000106[1] 1740485029.235749 mapwrite 0x5825fb thru 0x583ba8 (0x15ae bytes) 000109[1] 1740485029.269599 mapread 0x534354 thru 0x535b7a (0x1827 bytes) 000110[1] 1740485029.277337 trunc from 0x9780da to 0x420a7b (0x55765e bytes) 000113[1] 1740485029.296744 write 0x1d7758 thru 0x1d8392 (0xc3b bytes) 000114[1] 1740485029.305178 mapwrite 0x36d4db thru 0x36ee24 (0x194a bytes) 000117[1] 1740485029.331064 mapread 0x32e664 thru 0x330543 (0x1ee0 bytes) 000120[0] 1740485029.339872 write 0x6ec0e8 thru 0x6ec8f4 (0x80d bytes) 000121[0] 1740485029.353499 write 0x1272a0 thru 0x128916 (0x1677 bytes) 000124[1] 1740485029.363604 mapwrite 0x3436d5 thru 0x344061 (0x98d bytes) 000127[1] 1740485029.418854 write 0x650062 thru 0x651221 (0x11c0 bytes) 000129[1] 1740485029.430700 read 0x03ea45 thru 0x0405c8 (0x1b84 bytes) 000133[0] 1740485029.451563 trunc from 0x6ec8f5 to 0x38ed72 (0x35db82 bytes) 000137[0] 1740485029.486881 mapread 0x1174c2 thru 0x118aea (0x1629 bytes) 000138[1] 1740485029.492053 mapwrite 0x3b01ba thru 0x3b0ed8 (0xd1f bytes) 000140[0] 1740485029.545745 read 0x1b2108 thru 0x1b362c (0x1525 bytes) 000142[0] 1740485029.605272 write 0x2a6d43 thru 0x2a747f (0x73d bytes) 000143[0] 1740485029.621260 mapwrite 0x06c65a thru 0x06d70f (0x10b6 bytes) 000144[1] 1740485029.650346 read 0x121743 thru 0x121ad0 (0x38e bytes) 000146[0] 1740485029.693325 write 0x68ca2b thru 0x68db14 (0x10ea bytes) 000147[0] 1740485029.697639 mapread 0x3c6da4 thru 0x3c6f19 (0x176 bytes) 000151[0] 1740485029.713752 mapwrite 0x055f0a thru 0x057891 (0x1988 bytes) 000152[0] 1740485029.754966 write 0x6a46df thru 0x6a490f (0x231 bytes) truncating to largest ever: 0x902a8a 000154[0] 1740485029.759514 trunc from 0x6a4910 to 0x902a89 (0x25e17a bytes) 000157[0] 1740485029.772638 read 0x79578a thru 0x796f21 (0x1798 bytes) truncating to largest ever: 0x9f0b0c 000159[1] 1740485029.788988 trunc from 0x902a8a to 0x9f0b0b (0xee082 bytes) 000160[0] 1740485029.809623 mapread 0x61570e thru 0x617346 (0x1c39 bytes) 000165[1] 1740485029.840288 write 0x424717 thru 0x426515 (0x1dff bytes) 000167[0] 1740485029.884549 mapread 0x832a25 thru 0x83452c (0x1b08 bytes) 000168[1] 1740485029.903825 mapread 0x26db5b thru 0x26f445 (0x18eb bytes) 000171[0] 1740485029.910936 mapwrite 0x33341f thru 0x334145 (0xd27 bytes) 000172[1] 1740485029.943415 write 0x08b7bf thru 0x08d1c8 (0x1a0a bytes) 000173[0] 1740485029.956537 trunc from 0x9f0b0c to 0x052b3f (0x99dfcc bytes) 000174[1] 1740485029.977377 read 0x02aa1d thru 0x02c486 (0x1a6a bytes) 000177[1] 1740485029.992280 mapwrite 0x104281 thru 0x105405 (0x1185 bytes) 000178[0] 1740485030.035781 mapread 0x004af1 thru 0x005ecd (0x13dd bytes) 000179[1] 1740485030.049001 trunc from 0x105406 to 0x7d7e66 (0x6d2a61 bytes) 000180[1] 1740485030.064577 read 0x0d20a2 thru 0x0d259e (0x4fd bytes) 000182[1] 1740485030.078893 mapread 0x2b5006 thru 0x2b50ad (0xa8 bytes) 000185[0] 1740485030.091759 read 0x6e2251 thru 0x6e22de (0x8e bytes) 000187[0] 1740485030.100044 mapwrite 0x8def5b thru 0x8e0aa5 (0x1b4b bytes) 000189[1] 1740485030.142214 trunc from 0x8e0aa6 to 0x30b23d (0x5d5868 bytes) 000190[0] 1740485030.159740 mapread 0x1e867c thru 0x1e9168 (0xaed bytes) 000192[1] 1740485030.171611 read 0x2b7a14 thru 0x2b9944 (0x1f31 bytes) 000194[0] 1740485030.204808 mapwrite 0x03d4de thru 0x03f4d2 (0x1ff5 bytes) 000195[0] 1740485030.234406 trunc from 0x30b23e to 0x44ff98 (0x144d5b bytes) 000197[1] 1740485030.247676 write 0x5a2137 thru 0x5a4041 (0x1f0b bytes) 000199[0] 1740485030.259027 write 0x244461 thru 0x244c14 (0x7b4 bytes) 000201[1] 1740485030.268280 read 0x190404 thru 0x191b91 (0x178e bytes) 000203[0] 1740485030.278258 read 0x163eb9 thru 0x164362 (0x4aa bytes) 000210[0] 1740485030.299188 mapwrite 0x300061 thru 0x300e34 (0xdd4 bytes) 000211[0] 1740485030.325942 mapread 0x2bdaed thru 0x2bebe4 (0x10f8 bytes) 000213[0] 1740485030.345113 trunc from 0x5a4042 to 0x65fe7b (0xbbe3a bytes) 000216[0] 1740485030.367197 read 0x51f7dc thru 0x520a47 (0x126c bytes) 000218[0] 1740485030.375251 write 0x0441ec thru 0x045cd6 (0x1aeb bytes) 000219[0] 1740485030.382730 write 0x44a4f6 thru 0x44a6d7 (0x1e2 bytes) 000221[1] 1740485030.389708 trunc from 0x65fe7c to 0x324144 (0x33bd37 bytes) 000223[1] 1740485030.409398 trunc from 0x324145 to 0x3aeb28 (0x8a9e4 bytes) 000228[0] 1740485030.422588 mapread 0x31a4ab thru 0x31bbe3 (0x1739 bytes) 000229[1] 1740485030.438310 mapread 0x009594 thru 0x0099fa (0x467 bytes) 000230[0] 1740485030.441743 mapread 0x0bdae8 thru 0x0bee75 (0x138e bytes) 000231[1] 1740485030.448364 read 0x08770d thru 0x087edd (0x7d1 bytes) 000233[0] 1740485030.479556 trunc from 0x3aeb29 to 0x736664 (0x387b3c bytes) 000237[0] 1740485030.511650 read 0x644f55 thru 0x6462d7 (0x1383 bytes) 000238[0] 1740485030.521293 mapread 0x71135c thru 0x712780 (0x1425 bytes) 000239[1] 1740485030.532824 write 0x851454 thru 0x851ee4 (0xa91 bytes) 000240[1] 1740485030.540748 mapread 0x6ced80 thru 0x6cfe73 (0x10f4 bytes) 000242[0] 1740485030.567803 read 0x612691 thru 0x61356c (0xedc bytes) 000248[0] 1740485030.584476 mapread 0x7a37e4 thru 0x7a4430 (0xc4d bytes) 000249[1] 1740485030.596262 trunc from 0x851ee5 to 0x00958c (0x848958 bytes) 000250[1] 1740485030.610264 mapwrite 0x668822 thru 0x669bf1 (0x13d0 bytes) 000258[1] 1740485030.643410 read 0x3089aa thru 0x30a516 (0x1b6d bytes) 000263[1] 1740485030.661702 write 0x76113a thru 0x763013 (0x1eda bytes) 000264[1] 1740485030.667174 read 0x57c233 thru 0x57c823 (0x5f1 bytes) 000265[1] 1740485030.669625 mapwrite 0x4730d9 thru 0x4743f8 (0x1320 bytes) 000270[1] 1740485030.694759 mapwrite 0x0371ce thru 0x038006 (0xe39 bytes) 000272[0] 1740485030.710489 write 0x34474e thru 0x346523 (0x1dd6 bytes) 000274[0] 1740485030.723703 read 0x13eab2 thru 0x13ecd9 (0x228 bytes) 000279[0] 1740485030.748245 mapread 0x6a0f34 thru 0x6a2535 (0x1602 bytes) 000283[0] 1740485030.775575 mapread 0x6f53f4 thru 0x6f6bc5 (0x17d2 bytes) 000285[0] 1740485030.779663 mapwrite 0x0b2375 thru 0x0b2e5b (0xae7 bytes) 000286[0] 1740485030.795104 trunc from 0x763014 to 0x3f9523 (0x369af0 bytes) 000287[1] 1740485030.803576 mapwrite 0x1160c9 thru 0x1162bf (0x1f7 bytes) 000288[0] 1740485030.823332 trunc from 0x3f9524 to 0x213dac (0x1e5777 bytes) 000289[1] 1740485030.830085 write 0x93ab02 thru 0x93b646 (0xb45 bytes) 000290[0] 1740485030.841057 trunc from 0x93b647 to 0x80ae20 (0x130826 bytes) 000294[1] 1740485030.849431 mapread 0x41e72f thru 0x41f02e (0x900 bytes) 000295[0] 1740485030.858566 mapread 0x0c0072 thru 0x0c018f (0x11e bytes) 000296[1] 1740485030.860749 trunc from 0x80ae21 to 0x61aa01 (0x1f041f bytes) 000297[0] 1740485030.874041 trunc from 0x61aa02 to 0x537ab5 (0xe2f4c bytes) 000299[1] 1740485030.893272 mapwrite 0x816f1e thru 0x81877f (0x1862 bytes) 000300[0] 1740485030.929020 trunc from 0x818780 to 0x24e969 (0x5c9e16 bytes) 000301[1] 1740485030.943916 mapwrite 0x112e8b thru 0x113650 (0x7c6 bytes) 000309[0] 1740485031.013039 write 0x1f7f15 thru 0x1f97e2 (0x18ce bytes) 000310[1] 1740485031.022144 mapwrite 0x551515 thru 0x551635 (0x121 bytes) 000315[1] 1740485031.074826 write 0x6d4b13 thru 0x6d534c (0x83a bytes) 000316[0] 1740485031.080728 mapread 0x108647 thru 0x108be4 (0x59e bytes) 000319[0] 1740485031.100957 mapwrite 0x3220bd thru 0x323c0a (0x1b4e bytes) 000324[0] 1740485031.144518 trunc from 0x6d534d to 0x818b13 (0x1437c7 bytes) 000329[1] 1740485031.171315 trunc from 0x818b14 to 0x8232e4 (0xa7d1 bytes) 000331[1] 1740485031.184326 mapwrite 0x7d02bb thru 0x7d0a7f (0x7c5 bytes) 000332[0] 1740485031.199984 mapwrite 0x066e99 thru 0x0688e0 (0x1a48 bytes) 000333[0] 1740485031.225490 read 0x5a3b62 thru 0x5a57a8 (0x1c47 bytes) 000334[1] 1740485031.229624 read 0x2f0373 thru 0x2f1084 (0xd12 bytes) 000339[1] 1740485031.236295 mapread 0x016234 thru 0x0180e1 (0x1eae bytes) 000342[0] 1740485031.242731 read 0x184633 thru 0x1859cb (0x1399 bytes) 000343[0] 1740485031.244406 mapwrite 0x705631 thru 0x706dbc (0x178c bytes) 000344[0] 1740485031.265309 mapwrite 0x24eb06 thru 0x24f767 (0xc62 bytes) 000346[1] 1740485031.296800 mapwrite 0x7ca046 thru 0x7cbae9 (0x1aa4 bytes) 000348[1] 1740485031.336626 mapwrite 0x3abe1a thru 0x3ad513 (0x16fa bytes) 000350[1] 1740485031.370786 mapread 0x3be407 thru 0x3bf019 (0xc13 bytes) 000351[1] 1740485031.374134 write 0x92ebae thru 0x92ef89 (0x3dc bytes) 000352[1] 1740485031.378518 write 0x0b472c thru 0x0b54ad (0xd82 bytes) 000353[1] 1740485031.382202 read 0x213e33 thru 0x2155d4 (0x17a2 bytes) 000354[1] 1740485031.384166 write 0x46ed4e thru 0x470b52 (0x1e05 bytes) 000355[1] 1740485031.392328 write 0x24a864 thru 0x24b3d2 (0xb6f bytes) 000356[1] 1740485031.399760 mapwrite 0x34ec62 thru 0x34ffd9 (0x1378 bytes) 000358[1] 1740485031.427840 mapwrite 0x95c722 thru 0x95e338 (0x1c17 bytes) 000359[1] 1740485031.451975 write 0x1009e2 thru 0x1015aa (0xbc9 bytes) 000362[1] 1740485031.460519 trunc from 0x95e339 to 0x8d38aa (0x8aa8e bytes) 000365[1] 1740485031.474839 write 0x1fe236 thru 0x1ffe32 (0x1bfd bytes) 000369[0] 1740485031.485210 mapread 0x1d8b54 thru 0x1d8c0f (0xbc bytes) 000370[0] 1740485031.497756 mapwrite 0x666a33 thru 0x6679a5 (0xf73 bytes) 000373[0] 1740485031.526198 write 0x549aa4 thru 0x54a107 (0x664 bytes) 000374[0] 1740485031.533252 mapwrite 0x9ca5ed thru 0x9cae53 (0x867 bytes) 000375[1] 1740485031.563343 write 0x01273e thru 0x012803 (0xc6 bytes) 000376[0] 1740485031.572878 mapread 0x07891e thru 0x079af2 (0x11d5 bytes) 000378[0] 1740485031.585013 mapwrite 0x34a643 thru 0x34bec7 (0x1885 bytes) 000380[1] 1740485031.618070 trunc from 0x9cae54 to 0x3257ba (0x6a5699 bytes) 000382[0] 1740485031.640334 write 0x3031a0 thru 0x3044c5 (0x1326 bytes) 000383[0] 1740485031.649884 trunc from 0x3257bb to 0x64d000 (0x327846 bytes) 000385[1] 1740485031.664402 mapread 0x64bccd thru 0x64cf4b (0x127f bytes) 000391[1] 1740485031.688830 write 0x01afa6 thru 0x01b194 (0x1ef bytes) 000392[1] 1740485031.698394 mapwrite 0x549267 thru 0x54af56 (0x1cf0 bytes) 000393[0] 1740485031.726820 write 0x3d4f3c thru 0x3d4f69 (0x2e bytes) 000394[0] 1740485031.738533 write 0x3aca49 thru 0x3ad719 (0xcd1 bytes) 000398[1] 1740485031.755877 mapwrite 0x100488 thru 0x101cf6 (0x186f bytes) 000399[1] 1740485031.786198 read 0x4955b9 thru 0x497448 (0x1e90 bytes) 000401[1] 1740485031.788748 write 0x0f218d thru 0x0f3a2c (0x18a0 bytes) 000403[0] 1740485031.797254 read 0x2b89f8 thru 0x2b96f5 (0xcfe bytes) 000404[1] 1740485031.806709 write 0x741bdc thru 0x7438f1 (0x1d16 bytes) 000414[0] 1740485031.827654 trunc from 0x7438f2 to 0x270cb8 (0x4d2c39 bytes) 000417[0] 1740485031.852955 mapwrite 0x76f386 thru 0x770d74 (0x19ef bytes) 000419[0] 1740485031.887252 trunc from 0x770d75 to 0x39ba2e (0x3d5346 bytes) 000421[1] 1740485031.899048 mapread 0x243673 thru 0x24415e (0xaec bytes) 000422[1] 1740485031.906651 trunc from 0x39ba2f to 0x358a4a (0x42fe4 bytes) 000425[0] 1740485031.925392 mapread 0x0e7f32 thru 0x0e8db5 (0xe84 bytes) 000429[0] 1740485031.941706 write 0x05da32 thru 0x05e46d (0xa3c bytes) 000430[0] 1740485031.950263 trunc from 0x358a4b to 0x6e2df1 (0x38a3a7 bytes) 000431[1] 1740485031.958449 trunc from 0x6e2df2 to 0x4c31d7 (0x21fc1a bytes) 000434[1] 1740485031.974345 mapwrite 0x87982c thru 0x87a431 (0xc06 bytes) 000435[0] 1740485032.004879 mapwrite 0x961668 thru 0x961edd (0x876 bytes) 000441[0] 1740485032.053210 write 0x1cbecd thru 0x1ccd3d (0xe71 bytes) 000442[1] 1740485032.060779 trunc from 0x961ede to 0x54ff5e (0x411f7f bytes) 000443[0] 1740485032.074357 write 0x4ae634 thru 0x4afc63 (0x1630 bytes) 000445[1] 1740485032.088697 write 0x75de5b thru 0x75df8f (0x135 bytes) 000447[1] 1740485032.109348 mapwrite 0x14079c thru 0x1408ee (0x153 bytes) 000449[0] 1740485032.138301 trunc from 0x75df90 to 0x5743e9 (0x1e9ba6 bytes) 000451[0] 1740485032.158273 mapwrite 0x7d7794 thru 0x7d7ee4 (0x751 bytes) 000454[1] 1740485032.206015 mapwrite 0x760009 thru 0x7608cd (0x8c5 bytes) 000455[0] 1740485032.244929 mapread 0x76d0b9 thru 0x76d3b6 (0x2fe bytes) 000456[0] 1740485032.250421 mapwrite 0x075a8c thru 0x076de7 (0x135c bytes) 000457[0] 1740485032.268218 mapread 0x703d91 thru 0x70547b (0x16eb bytes) 000458[0] 1740485032.273716 mapwrite 0x18efd3 thru 0x18f409 (0x437 bytes) 000459[0] 1740485032.293277 mapread 0x7c2a5d thru 0x7c355c (0xb00 bytes) 000460[1] 1740485032.302932 mapwrite 0x6db0c4 thru 0x6dca1e (0x195b bytes) 000472[1] 1740485032.412629 trunc from 0x7d7ee5 to 0x070383 (0x767b61 bytes) 000473[0] 1740485032.424789 read 0x05dad0 thru 0x05ef9c (0x14cd bytes) 000474[0] 1740485032.431808 read 0x068397 thru 0x068c0f (0x879 bytes) 000475[1] 1740485032.439566 mapwrite 0x7c305a thru 0x7c35ac (0x553 bytes) 000479[0] 1740485032.487312 mapread 0x327c80 thru 0x328d7b (0x10fc bytes) 000480[0] 1740485032.505157 trunc from 0x7c35ad to 0x82ec36 (0x6b68a bytes) 000481[0] 1740485032.527279 mapwrite 0x26b6e1 thru 0x26ca67 (0x1387 bytes) 000482[0] 1740485032.554668 mapread 0x79300d thru 0x7943c1 (0x13b5 bytes) 000483[1] 1740485032.557978 mapwrite 0x181a32 thru 0x182457 (0xa26 bytes) 000484[1] 1740485032.594227 trunc from 0x82ec37 to 0x4c8fe4 (0x365c52 bytes) 000486[0] 1740485032.613191 mapread 0x10cf53 thru 0x10d876 (0x924 bytes) 000489[1] 1740485032.631203 write 0x5fd656 thru 0x5ff41e (0x1dc9 bytes) 000490[1] 1740485032.635612 mapread 0x3d954f thru 0x3d9906 (0x3b8 bytes) 000491[0] 1740485032.641306 mapread 0x5e31ac thru 0x5e4d7a (0x1bcf bytes) 000492[1] 1740485032.651353 write 0x5cb320 thru 0x5cc5d4 (0x12b5 bytes) 000494[1] 1740485032.660957 write 0x2ffc0a thru 0x300a4b (0xe42 bytes) 000496[1] 1740485032.671224 mapwrite 0x21a5e3 thru 0x21b873 (0x1291 bytes) 000497[1] 1740485032.709913 mapread 0x1be93f thru 0x1bef1f (0x5e1 bytes) 000498[1] 1740485032.712933 read 0x14852f thru 0x14a043 (0x1b15 bytes) 000499[0] 1740485032.715112 trunc from 0x5ff41f to 0x787547 (0x188129 bytes) All operations completed A-OK! PASS 16b (7s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 07:03:54 (1740485034) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 07:03:55 (1740485035) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.16269 s, 51.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.147732 s, 56.8 MB/s ldlm.namespaces.MGC192.168.201.153@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b581e800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58d8000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b581e800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b58d8000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b581e800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b58d8000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.19201 s, 25.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.995524 s, 105 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.693077 s, 151 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.99064 s, 52.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.26584 s, 32.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.05916 s, 99.0 MB/s PASS 16d (16s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 07:04:11 (1740485051) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0779554 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472916 s, 22.2 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 07:04:13 (1740485053) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 07:04:36 (1740485076) TIMEOUT PASS 16g (22s) == sanityn test 16h: mmap read after truncate file ======= 07:04:58 (1740485098) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) == sanityn test 16i: read after truncate file ============ 07:05:01 (1740485101) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 16j: race dio with buffered i/o ========== 07:05:03 (1740485103) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.478436 s, 54.8 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.324613 s, 80.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.736031 s, 35.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.202411 s, 130 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.598815 s, 43.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.220984 s, 119 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.927511 s, 28.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.446371 s, 58.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.523589 s, 50.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.933822 s, 28.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.989547 s, 26.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.273482 s, 95.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.356419 s, 73.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.786935 s, 33.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.824902 s, 31.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.271394 s, 96.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.379154 s, 69.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.975796 s, 26.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 1.05529 s, 24.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (11s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 07:05:14 (1740485114) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x32cce75 skipping zero size read truncating to largest ever: 0x3c569db truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Tue Feb 25 07:05:25 EST 2025 All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Tue Feb 25 07:05:25 EST 2025 + fsx 3 done: rc=0 Tue Feb 25 07:05:25 EST 2025 + fsx 4 done: rc=0 Tue Feb 25 07:05:25 EST 2025 + fsx 5 done: rc=0 Tue Feb 25 07:05:25 EST 2025 PASS 16k (13s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 07:05:27 (1740485127) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 07:05:31 (1740485131) mmap test1: basic mmap operation (PASS, 0.02269s) mmap test2: MAP_PRIVATE not write back (PASS, 0.019406s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.3988s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.0891s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.043897s) mmap test6: check mmap write/read content on two nodes (PASS, 0.051455s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.35254s) mmap test11: multi-thread shared mmap (PASS, 0.33639s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0169s) PASS 18 (11s) == sanityn test 19: test concurrent uncached read races ========================================================================= 07:05:42 (1740485142) oleg153-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg153-client: oleg153-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 07:05:43 (1740485143) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 07:05:45 (1740485145) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 07:05:47 (1740485147) now is 1740485147 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 new atime is 1740485208 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 07:06:50 (1740485210) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 32.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118066 402 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117231 399 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre[MDT:0] filesystem_summary: 502242 386 501856 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118066 402 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117231 399 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 07:06:52 (1740485212) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 07:06:54 (1740485214) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 07:06:56 (1740485216) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 07:06:57 (1740485217) utime: good mknod times 1740485216 <= 1740485217 <= 1740485217 for /mnt/lustre/f26a utime: good mknod times 1740485216 <= 1740485217 <= 1740485217 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 07:07:00 (1740485220) PASS 26b (3s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 07:07:03 (1740485223) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-02-25 07:07:04.000000000 -0500 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-02-25 07:07:04.000000000 -0500 Birth: - PASS 26c (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 07:07:07 (1740485227) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 07:07:11 (1740485231) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 07:07:14 (1740485234) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg153-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.153@tcp:/lustre 7542784 38912 7499776 1% /mnt/lustre2 PASS 31b (19s) == sanityn test 31r: open-rename(replace) race =========== 07:07:33 (1740485253) -rw-r--r-- 1 root root 0 Feb 25 07:07 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 07:07:38 (1740485258) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13523 Terminated tail -f $tfile PASS 31s (2s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 07:07:40 (1740485260) PASS 31t (3s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 07:07:43 (1740485263) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 07:07:44 (1740485264) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 07:07:45 (1740485265) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (0s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 07:07:45 (1740485265) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 07:07:46 (1740485266) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b581e800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b581e800.ost_server_uuid in FULL state after 0 sec oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b581e800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b581e800.ost_server_uuid in IDLE state after 0 sec oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b581e800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b581e800.ost_server_uuid in FULL state after 0 sec oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b581e800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b581e800.ost_server_uuid in IDLE state after 0 sec oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b581e800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b581e800.ost_server_uuid in FULL state after 0 sec oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b581e800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b581e800.ost_server_uuid in IDLE state after 0 sec PASS 34 (88s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 07:09:14 (1740485354) fail_loc=0x80000317 Race attempt 0 Wait for 20166 20305 for 60 sec... - open/close 3868 (time 1740485366.06 total 10.00 last 386.76) total: 4000 open/close in 10.29 seconds: 388.75 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.153@tcp:/lustre 7.2G 38M 7.2G 1% /mnt/lustre 192.168.201.153@tcp:/lustre 7.2G 38M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 07:10:18 (1740485418) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg153-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.329009 s, 159 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7499776) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.276986 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.292841 s, 179 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7497728) after_dd(7448576) after(7499776) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.29163 s, 180 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7499776) after_dd(7448576) after(7499776) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.333298 s, 157 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7499776) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.265351 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7495680) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.259487 s, 202 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7497728) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.277577 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7495680) after_dd(7448576) after(7499776) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.266821 s, 196 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7499776) after_dd(7448576) after(7499776) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.270768 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7499776) after_dd(7448576) after(7499776) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.273651 s, 192 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7499776) after_dd(7446528) after(7497728) PASS 36 (224s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 07:14:02 (1740485642) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 total: 10000 create in 9.95 seconds: 1005.52 ops/second PASS 37 (13s) == sanityn test 39a: file mtime does not change after rename ========================================================== 07:14:15 (1740485655) repeat after cancel_lru_locks PASS 39a (1s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 07:14:16 (1740485656) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 07:14:19 (1740485659) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 07:14:22 (1740485662) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 07:14:23 (1740485663) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 07:14:30 (1740485670) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 07:14:38 (1740485678) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 07:14:45 (1740485685) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 07:14:52 (1740485692) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 07:14:58 (1740485698) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 07:15:02 (1740485702) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 07:15:06 (1740485706) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 07:15:11 (1740485711) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 07:15:15 (1740485715) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 07:15:19 (1740485719) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 07:15:23 (1740485723) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 07:15:27 (1740485727) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 07:15:32 (1740485732) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (648s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 07:26:20 (1740486380) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 07:26:25 (1740486385) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 07:26:29 (1740486389) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 07:26:33 (1740486393) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 07:26:37 (1740486397) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 07:26:42 (1740486402) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 07:26:46 (1740486406) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 07:26:50 (1740486410) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 07:26:55 (1740486415) PASS 43a (12s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 07:27:07 (1740486427) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 07:27:11 (1740486431) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 07:27:15 (1740486435) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 07:27:20 (1740486440) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 07:27:24 (1740486444) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 07:27:28 (1740486448) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 07:27:32 (1740486452) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 07:27:37 (1740486457) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 07:27:37 (1740486457) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (34s) == sanityn test 43k: unlink vs create ==================== 07:28:11 (1740486491) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (361s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 07:34:12 (1740486852) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Feb 25 07:34:15 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 07:34:16 (1740486856) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 07:34:20 (1740486860) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 07:34:25 (1740486865) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 07:34:29 (1740486869) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 07:34:33 (1740486873) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 07:34:37 (1740486877) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 07:34:41 (1740486881) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 07:34:46 (1740486886) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 07:34:47 (1740486887) PASS 45a (21s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 07:35:08 (1740486908) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 07:35:12 (1740486912) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 07:35:16 (1740486916) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 07:35:21 (1740486921) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 07:35:25 (1740486925) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 07:35:29 (1740486929) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 07:35:33 (1740486933) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 07:35:37 (1740486937) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 07:35:38 (1740486938) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (376s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 07:41:54 (1740487314) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 07:41:59 (1740487319) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 07:42:03 (1740487323) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 07:42:08 (1740487328) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 07:42:13 (1740487333) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 07:42:17 (1740487337) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 07:42:21 (1740487341) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 07:42:26 (1740487346) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 07:42:31 (1740487351) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 07:42:32 (1740487352) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 07:42:33 (1740487353) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 07:42:34 (1740487354) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 07:42:35 (1740487355) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 07:42:36 (1740487356) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 07:42:37 (1740487357) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 07:42:38 (1740487358) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 07:42:39 (1740487359) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00706436 s, 1.4 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 07:42:46 (1740487366) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00565748 s, 28.1 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3519: kill: (10976) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 07:42:50 (1740487370) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00217251 s, 471 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213049 s, 49.2 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 07:43:03 (1740487383) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00375719 s, 273 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 07:43:09 (1740487389) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011029 s, 95.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 07:43:12 (1740487392) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2453 0x995 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2453 0x995 0x240000400 read lease(1) has applied. PASS 51e (3s) == sanityn test 54: rename locking ======================= 07:43:15 (1740487395) llite.lustre-ffff8800b581e800.xattr_cache=1 llite.lustre-ffff8800b58d8000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 07:43:16.000000000 -0500 Modify: 2025-02-25 07:43:16.000000000 -0500 Change: 2025-02-25 07:43:16.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 07:43:21.000000000 -0500 Modify: 2025-02-25 07:43:21.000000000 -0500 Change: 2025-02-25 07:43:21.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 07:43:26.000000000 -0500 Modify: 2025-02-25 07:43:26.000000000 -0500 Change: 2025-02-25 07:43:26.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 07:43:26.000000000 -0500 Modify: 2025-02-25 07:43:27.000000000 -0500 Change: 2025-02-25 07:43:27.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 07:43:32.000000000 -0500 Modify: 2025-02-25 07:43:32.000000000 -0500 Change: 2025-02-25 07:43:32.000000000 -0500 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 07:43:32.000000000 -0500 Modify: 2025-02-25 07:43:32.000000000 -0500 Change: 2025-02-25 07:43:32.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 07:43:39 (1740487419) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 07:43:46 (1740487426) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 07:43:53 (1740487433) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 07:44:05 (1740487445) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (8s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 07:44:13 (1740487453) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1740487454 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 3.39106 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (20s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 07:44:33 (1740487473) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1740487474 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.20946 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (33s) == sanityn test 60: Verify data_version behaviour ======== 07:45:06 (1740487506) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 07:45:08 (1740487508) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 07:45:09 (1740487509) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 07:45:11 (1740487511) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00312277 s, 21.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 07:45:12 (1740487512) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00359576 s, 18.2 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 07:45:13 (1740487513) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 07:45:14 (1740487514) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00358945 s, 18.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (0s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 07:45:14 (1740487514) llite.lustre-ffff8800b581e800.xattr_cache=1 llite.lustre-ffff8800b58d8000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 07:45:16 (1740487516) llite.lustre-ffff8800b581e800.xattr_cache=1 llite.lustre-ffff8800b58d8000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 07:45:18 (1740487518) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00220451 s, 465 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00218168 s, 469 kB/s 23397: get lock2 23398: get lock1 23398: done 23398: get lock2 23397: done 23397: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 23397: cannot get lock1: Resource deadlock avoided 23398: done 23398: exit rc=0 23397: done 23397: exit rc=0 PASS 74 (5s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 07:45:23 (1740487523) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0130634 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891228 s, 118 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4108: 24039 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 07:45:30 (1740487530) opening files: ........ closing files: ........ PASS 76 (14s) == sanityn test 77a: check FIFO NRS policy =============== 07:45:44 (1740487544) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0933683 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00783372 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935978 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968177 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237715 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134117 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324473 s, 32.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354829 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379199 s, 27.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0359264 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295531 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311834 s, 33.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0324792 s, 32.3 MB/s , 0.0346884 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447869 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485224 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488849 s, 21.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b581e800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764532 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079691 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00643464 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787067 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913931 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101119 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114855 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090525 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904066 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872667 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987846 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707059 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0102552 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.00954675 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094347 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105216 s, 99.7 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 07:45:46 (1740487546) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0833138 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985644 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990203 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262113 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288416 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228267 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394664 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278445 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358799 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0499584 s, 21.0 MB/s , 0.0233219 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0517429 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047494 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535177 s, 19.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0520613 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532665 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501291 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008949 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876531 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127759 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014003 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966288 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101838 s, 103 MB/s , 0.0102449 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0116962 s, 89.7 MB/s , 0.0119282 s, 87.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0126519 s, 82.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0118526 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129686 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136034 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144045 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144687 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164279 s, 63.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0861933 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802786 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143138 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017465 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193343 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124371 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088148 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193379 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123975 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154431 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453365 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405035 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353373 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392761 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289589 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0548737 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567428 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00584843 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00532029 s, 197 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00669978 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731259 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748676 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185145 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179932 s, 58.3 MB/s , 0.018037 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184262 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779227 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811667 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826571 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958961 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978378 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089251 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102343 s, 102 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 07:45:49 (1740487549) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0825573 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986772 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114433 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269656 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297648 s, 35.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368666 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398851 s, 26.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0373785 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384153 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400349 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412348 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419366 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451064 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411527 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401545 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438668 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497176 s, 21.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00959962 s, 109 MB/s , 0.00160776 s, 652 MB/s , 0.00785041 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008496 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100416 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107538 s, 97.5 MB/s , 0.0103229 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107576 s, 97.5 MB/s , 0.011658 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0119874 s, 87.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0112574 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127093 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141941 s, 73.9 MB/s , 0.0131449 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131938 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142252 s, 73.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0840684 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686578 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152496 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152917 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0568703 s, 18.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0524731 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057691 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0566675 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0628984 s, 16.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0679497 s, 15.4 MB/s , 0.0661142 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0644531 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0660487 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0656949 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0734331 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0690235 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0765366 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980983 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00647471 s, 162 MB/s , 0.00648365 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00675949 s, 155 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00743484 s, 141 MB/s , 0.00988764 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810114 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801658 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968226 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010769 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108203 s, 96.9 MB/s , 0.0106638 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102926 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119924 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121277 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120087 s, 87.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0932013 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116517 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148962 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035325 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324925 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405617 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455216 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352818 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376245 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473151 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046617 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476766 s, 22.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480093 s, 21.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0449206 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471206 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448694 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463932 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881441 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823824 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103644 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847485 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915854 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101152 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099086 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00546639 s, 192 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976588 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00632317 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965862 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109053 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108144 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113727 s, 92.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127428 s, 82.3 MB/s , 0.0110909 s, 94.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 07:45:53 (1740487553) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0912061 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102597 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124149 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204443 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020536 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250845 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209632 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024955 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189473 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312429 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389552 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431321 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461894 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391526 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467617 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415708 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511138 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127968 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802617 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897011 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854099 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013171 s, 79.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114027 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110656 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119588 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116964 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179382 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124727 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114567 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120076 s, 87.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0129632 s, 80.9 MB/s , 0.0130145 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147501 s, 71.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.116589 s, 144 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723999 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0104772 s, 100 MB/s , 0.00801049 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108436 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847149 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0727676 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0653939 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0649228 s, 16.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0669329 s, 15.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0688202 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0803354 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0800382 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0720421 s, 14.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0732282 s, 14.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0839501 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0907736 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00797149 s, 132 MB/s 1048576 bytes (1.0 MB) copied, 0.00724444 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00717262 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101321 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867319 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846685 s, 124 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00866335 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00988192 s, 106 MB/s , 0.00876045 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925666 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939738 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102798 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0110242 s, 95.1 MB/s 1048576 bytes (1.0 MB) copied, 0.010095 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112649 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133668 s, 78.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0902006 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00604083 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107724 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117908 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152723 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493509 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420369 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554212 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495394 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473322 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0591591 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0615893 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506969 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0661102 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0546707 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0575806 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594571 s, 17.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00564193 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143642 s, 73.0 MB/s , 0.0162489 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144478 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875455 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016658 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073997 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.004566 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838597 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906193 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822669 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888347 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868359 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008828 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104072 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975656 s, 107 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 07:45:56 (1740487556) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.53@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0751122 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110026 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135241 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322489 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297902 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0257512 s, 40.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0341865 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260869 s, 40.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346902 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372679 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311283 s, 33.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0395014 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430634 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348859 s, 30.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0419024 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334863 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459555 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779371 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190081 s, 552 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710607 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00747649 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109753 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00426097 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097816 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105455 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103991 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108867 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105179 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122266 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112154 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119445 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125259 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124098 s, 84.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0845308 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108249 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140365 s, 74.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353514 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033479 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363536 s, 28.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0354005 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364947 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360404 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398639 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037978 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440036 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372847 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437308 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446717 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474388 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474034 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00682201 s, 154 MB/s , 0.0100564 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00623408 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110996 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00732452 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00443846 s, 236 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742896 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968717 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00914445 s, 115 MB/s , 0.00874186 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125822 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099955 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105611 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105567 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010453 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120947 s, 86.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0982305 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841222 s, 125 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132989 s, 78.8 MB/s 1048576 bytes (1.0 MB) copied, 0.018384 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181559 s, 57.8 MB/s , 0.0194191 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188299 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151647 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129864 s, 80.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109833 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014483 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0676353 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0609125 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.061248 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0526121 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0724648 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0652105 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0070683 s, 148 MB/s 1+0 records in 1+0 records out , 0.0070269 s, 149 MB/s 1048576 bytes (1.0 MB) copied, 0.00706971 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725387 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00857876 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00858894 s, 122 MB/s , 0.0137645 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156785 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106478 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113314 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110927 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011974 s, 87.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0137939 s, 76.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0136414 s, 76.9 MB/s , 0.0139824 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157274 s, 66.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0841467 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999465 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114236 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118018 s, 88.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330789 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375139 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374754 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406774 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324941 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436265 s, 24.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0443445 s, 23.6 MB/s , 0.0408131 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417316 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420617 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380277 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459047 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449981 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846377 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802767 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712936 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825734 s, 127 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00884188 s, 119 MB/s , 0.0075603 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011686 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010611 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892027 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115422 s, 90.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00896417 s, 117 MB/s , 0.00937665 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983877 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976472 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946134 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110948 s, 94.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 07:46:02 (1740487562) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.089609 s, 187 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318303 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283634 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0631609 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0646175 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0670558 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639892 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0701742 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697177 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0730743 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0727469 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0777818 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828001 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828199 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0811349 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0820653 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0993187 s, 10.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952906 s, 110 MB/s , 0.00945112 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972589 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112114 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157491 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148421 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143715 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014432 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149072 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148096 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159065 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171546 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163768 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165275 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017999 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171149 s, 61.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107991 s, 155 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911269 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129822 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719277 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0760686 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0750262 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0760607 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0801629 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.08085 s, 13.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0811081 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0806966 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0755104 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0786178 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0837338 s, 12.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0847953 s, 12.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0831492 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0893848 s, 11.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00894931 s, 117 MB/s , 0.0139297 s, 75.3 MB/s , 0.011965 s, 87.6 MB/s , 0.00994053 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122924 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137135 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152174 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014035 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012937 s, 81.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0129066 s, 81.2 MB/s , 0.0129561 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141592 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141835 s, 73.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.015904 s, 65.9 MB/s , 0.0153826 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149286 s, 70.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0963036 s, 174 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106837 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139528 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155835 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370407 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335613 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478553 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363272 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527519 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0522934 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493724 s, 21.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0632701 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0710345 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0642308 s, 16.3 MB/s 1048576 bytes (1.0 MB) copied, 0.058595 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0598069 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0771352 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195396 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198053 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980545 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924188 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0211157 s, 49.7 MB/s , 0.00983086 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975841 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115958 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117005 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111654 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116357 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127221 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117375 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122709 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132536 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146347 s, 71.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0823284 s, 204 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106507 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136479 s, 76.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224995 s, 46.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0259682 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0213783 s, 49.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0276742 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320792 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287521 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339347 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405632 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439682 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398919 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467982 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440563 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479684 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046435 s, 22.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176032 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170149 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008589 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890271 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109378 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978876 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113495 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204973 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178561 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116728 s, 89.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0136373 s, 76.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0121763 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122365 s, 85.7 MB/s , 0.0124422 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012254 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132689 s, 79.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 07:46:08 (1740487568) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 07:46:11 (1740487571) oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg153-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg153-client: oleg153-server: ssh exited with exit code 2 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg153-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg153-client: oleg153-server: ssh exited with exit code 12 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg153-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg153-client: oleg153-server: ssh exited with exit code 12 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg153-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg153-client: oleg153-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 07:46:14 (1740487574) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (5s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 07:46:19 (1740487579) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0856825 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782336 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933352 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175263 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150898 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240385 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247269 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280203 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267415 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331751 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258425 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258461 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371922 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285618 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359919 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262244 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315518 s, 33.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00783474 s, 134 MB/s , 0.00993407 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884403 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695101 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922998 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0093589 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921046 s, 114 MB/s , 0.00851413 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010309 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918444 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106933 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01039 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101229 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985639 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117208 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127695 s, 82.1 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91581 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4318 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (50s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 07:47:09 (1740487629) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.2169 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176434 s, 59.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0156534 s, 67.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165967 s, 63.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0278685 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278099 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100744 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0235407 s, 44.5 MB/s , 0.0179685 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178552 s, 5.9 MB/s , 0.177576 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.187187 s, 5.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.172084 s, 6.1 MB/s 1048576 bytes (1.0 MB) copied, 0.175476 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190371 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191962 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.193672 s, 5.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111518 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116681 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854005 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120677 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013553 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142366 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117875 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105121 s, 99.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155717 s, 67.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0134141 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011772 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128409 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121743 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123651 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128776 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014466 s, 72.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0287 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9681 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.446039 s, 37.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0161242 s, 65.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0155229 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180732 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205716 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198012 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00824115 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909824 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112251 s, 93.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.146732 s, 7.1 MB/s , 0.144523 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154203 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135434 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167507 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147153 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169922 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148035 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00857141 s, 122 MB/s , 0.00853394 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951927 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914273 s, 115 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00473231 s, 222 MB/s 1048576 bytes (1.0 MB) copied, 0.00485899 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119732 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106866 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966271 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869996 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00757403 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126953 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109272 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010595 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127387 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131714 s, 79.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0375 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9449 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (130s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 07:49:19 (1740487759) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22732 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014446 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111337 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250018 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198504 s, 52.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.041052 s, 25.5 MB/s , 0.0268616 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308978 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408073 s, 25.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.173468 s, 6.0 MB/s , 0.173479 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.173062 s, 6.1 MB/s , 0.189334 s, 5.5 MB/s 1048576 bytes (1.0 MB) copied, 0.186655 s, 5.6 MB/s 1048576 bytes (1.0 MB) copied, 0.18314 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178678 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.194625 s, 5.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963223 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944447 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00649947 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105353 s, 99.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968054 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918226 s, 114 MB/s 1048576 bytes (1.0 MB) copied, 0.00856502 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985944 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122425 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119916 s, 87.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0112793 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110763 s, 94.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0118688 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139615 s, 75.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0105202 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146934 s, 71.4 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8343 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9613 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg153-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22868 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120359 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189091 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258319 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163987 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353102 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189781 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227697 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232982 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15285 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15731 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171415 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167098 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173391 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169649 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.211049 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.204595 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961777 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101225 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0102299 s, 103 MB/s , 0.0110423 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120798 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106537 s, 98.4 MB/s , 0.0116801 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128233 s, 81.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113403 s, 92.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0115878 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122543 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012461 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118401 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110264 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137138 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130621 s, 80.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.835 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9676 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (130s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 07:51:29 (1740487889) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.240639 s, 69.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085659 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239818 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201877 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240556 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321051 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0225324 s, 46.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0257738 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382278 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303318 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481092 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441566 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407661 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0551706 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0572655 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055991 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0676164 s, 15.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849752 s, 123 MB/s 1+0 records in 1+0 records out , 0.00862709 s, 122 MB/s 1048576 bytes (1.0 MB) copied, 0.00628191 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0064882 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773488 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011474 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00443286 s, 237 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00146091 s, 718 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918663 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00922148 s, 114 MB/s , 0.010138 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101065 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108277 s, 96.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00930003 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972368 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115992 s, 90.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91406 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73413 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.53@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.53@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.116797 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712567 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010069 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123273 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146634 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199293 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132731 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140572 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163529 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126545 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00630882 s, 166 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.204321 s, 5.1 MB/s , 0.212504 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.236932 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.243584 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.294269 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.298905 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00607148 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00624772 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915907 s, 114 MB/s , 0.00631168 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759974 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00762749 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866866 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121805 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943726 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100394 s, 104 MB/s , 0.0100164 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104094 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109878 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114462 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124637 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126418 s, 82.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96882 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.749 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.53@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0916874 s, 183 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922045 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154225 s, 68.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176963 s, 59.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0193685 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187897 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209603 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209407 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322428 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284175 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136141 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230592 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247855 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028359 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234435 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327887 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037976 s, 27.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107188 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817077 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839127 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998934 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938762 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115888 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983086 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969984 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991075 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963356 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996707 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104183 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101273 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108378 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105263 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111535 s, 94.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94469 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87869 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0903101 s, 186 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819374 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779482 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0157628 s, 66.5 MB/s , 0.0248436 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233735 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285602 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172592 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018936 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164196 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0318421 s, 32.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0401583 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419049 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12857 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134878 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129709 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142105 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0059008 s, 178 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0084491 s, 124 MB/s , 0.010017 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133362 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165973 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00615714 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695912 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898499 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898588 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985523 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102576 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112927 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124013 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148558 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154518 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158874 s, 66.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.8047 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74063 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92448 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73558 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.152388 s, 110 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170039 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281292 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216828 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798148 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288006 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313175 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213883 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197776 s, 53.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0633783 s, 16.5 MB/s , 0.067173 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597068 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0590749 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0794413 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0806792 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0550458 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0579641 s, 18.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686741 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725164 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00747769 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934409 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00650206 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00857383 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00820332 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748935 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00127465 s, 823 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913648 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977562 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890475 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940228 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00956974 s, 110 MB/s , 0.00973598 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950156 s, 110 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7124 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4381 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.125781 s, 133 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103093 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.012541 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967693 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151052 s, 69.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239264 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215272 s, 48.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0215123 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255003 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246328 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270522 s, 38.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215173 s, 48.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0260107 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290393 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164566 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248048 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174217 s, 60.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182566 s, 57.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0125992 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169139 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144958 s, 72.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0142523 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192546 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159189 s, 65.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0162298 s, 64.6 MB/s , 0.0181271 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165654 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180552 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180395 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175587 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180897 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178839 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189628 s, 55.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93917 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73851 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0831832 s, 202 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100318 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203193 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121446 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225339 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253054 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194249 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178594 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212679 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295966 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0416226 s, 25.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0351009 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042358 s, 24.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.126408 s, 8.3 MB/s , 0.133878 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126693 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129497 s, 8.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742281 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680085 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00675093 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878783 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00320268 s, 327 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00830968 s, 126 MB/s , 0.00907791 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950961 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963816 s, 109 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00967081 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00776534 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788335 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836276 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834138 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852918 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126852 s, 82.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77543 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74583 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79439 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73374 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (357s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 07:57:26 (1740488246) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 07:57:30 (1740488250) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg153-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.05525 s, 343 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg153-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0139 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 07:58:20 (1740488300) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.133014 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016636 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155562 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237009 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426593 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449839 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0450564 s, 23.3 MB/s 1+0 records in 1+0 records out , 0.0416163 s, 25.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0441996 s, 23.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0597392 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568791 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589925 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0531911 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597812 s, 17.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480984 s, 21.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0606817 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0708643 s, 14.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b581e800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00946173 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.0120599 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117611 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825477 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136179 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106658 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102686 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114311 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126932 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113178 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132229 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112601 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0124598 s, 84.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0127029 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119847 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00364475 s, 288 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.00128 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97597 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.249776 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182608 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203108 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214156 s, 49.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0278442 s, 37.7 MB/s , 0.0313048 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482843 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445527 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0565284 s, 18.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524927 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.058115 s, 18.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394509 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396762 s, 26.4 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0552422 s, 19.0 MB/s , 0.0479612 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.244609 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.249215 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169066 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128722 s, 81.5 MB/s , 0.013328 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146796 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020687 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155371 s, 67.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0160158 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01694 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188722 s, 55.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0181877 s, 57.7 MB/s , 0.0181502 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180355 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224722 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215355 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023067 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282014 s, 37.2 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg153-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg153-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96602 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96914 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (64s) == sanityn test 77o: Changing rank should not panic ====== 07:59:24 (1740488364) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 07:59:27 (1740488367) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (27s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 07:59:54 (1740488394) mds.MDS.mdt.nrs_policies=tbf oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg153-client: oleg153-server: ssh exited with exit code 36 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg153-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg153-client: oleg153-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 08:00:07 (1740488407) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 21351 kill md_thread xargs PASS 77r (37s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 08:00:44 (1740488444) oleg153-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg153-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg153-client: oleg153-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 08:00:48 (1740488448) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506740 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 08:00:48.000000000 -0500 Modify: 2025-02-25 08:00:48.000000000 -0500 Change: 2025-02-25 08:00:48.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 08:01:00 (1740488460) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 08:01:02 (1740488462) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 08:01:03 (1740488463) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 08:01:04 (1740488464) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 08:01:05 (1740488465) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 08:01:06 (1740488466) - open/close 2456 (time 1740488478.44 total 10.00 last 245.54) total: 2500 open/close in 10.19 seconds: 245.38 ops/second snapshot_time 1740488491.929054882 secs.nsecs start_time 1740488467.273162800 secs.nsecs elapsed_time 24.655892082 secs.nsecs open 2505 samples [usecs] 54 2664 1148376 549835468 close 2504 samples [usecs] 61 304 357416 53106914 mknod 2501 samples [usecs] 159 2591 1023519 438992195 unlink 2500 samples [usecs] 269 12059 1036708 1358249040 mkdir 2 samples [usecs] 898 1077 1975 1966333 rmdir 2 samples [usecs] 1581 2518 4099 8839885 rename 2500 samples [usecs] 213 11930 2717036 5326680886 getattr 3761 samples [usecs] 5 196 51358 1041720 parallel_rename_file 2500 samples [usecs] 213 11930 2717036 5326680886 crossdir_rename 2500 samples [usecs] 213 11930 2717036 5326680886 PASS 81d (27s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 08:01:33 (1740488493) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6990 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6990 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 08:01:35 (1740488495) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 08:01:36 (1740488496) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 85: Lustre API root cache race =========== 08:01:44 (1740488504) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1740488504 Finishing test test1 at 1740488505 PASS 85 (3s) == sanityn test 90: open/create and unlink striped directory ========================================================== 08:01:47 (1740488507) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 08:01:48 (1740488508) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 08:01:49 (1740488509) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 08:01:50 (1740488510) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6092 0x17cc 0x240000400 1 6075 0x17bb 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6093 0x17cd 0x240000400 1 6076 0x17bc 0x280000400 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 08:02:00 (1740488520) osc.lustre-OST0000-osc-ffff8800b581e800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b58d8000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b581e800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b58d8000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0311749 s, 263 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00775 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5744: 9766 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0228826 s, 179 kB/s osc.lustre-OST0000-osc-ffff8800b581e800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b58d8000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b581e800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58d8000.idle_timeout=debug PASS 94 (11s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 08:02:10 (1740488530) llite.lustre-ffff8800b581e800.fast_read=0 llite.lustre-ffff8800b58d8000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0203384 s, 103 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800b581e800.fast_read=1 llite.lustre-ffff8800b58d8000.fast_read=1 PASS 95a (19s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 08:02:29 (1740488549) llite.lustre-ffff8800b581e800.fast_read=0 llite.lustre-ffff8800b58d8000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00295937 s, 4.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b581e800.fast_read=1 llite.lustre-ffff8800b58d8000.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 08:02:38 (1740488558) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 08:02:39 (1740488559) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00271851 s, 1.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 08:02:41 (1740488561) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0200444 s, 105 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 08:02:44 (1740488564) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0351204 s, 59.7 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 08:02:46 (1740488566) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00292343 s, 4.1 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 08:02:49 (1740488569) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00082399 s, 5.0 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 08:02:51 (1740488571) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00082961 s, 4.9 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 08:02:53 (1740488573) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000925001 s, 4.4 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 08:02:57 (1740488577) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1740488578 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1740488581 Finishing test test23 at 1740488581 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 08:03:05 (1740488585) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 08:03:06 (1740488586) fail_loc=0x416 PASS 105 (22s) == sanityn test 106a: Verify the btime via statx() ======= 08:03:29 (1740488609) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 08:03:30 (1740488610) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 08:03:31 (1740488611) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 08:03:32 (1740488612) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0738109 s, 142 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6081 0x17c1 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 08:03:36 (1740488616) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.07308 s, 143 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6101 0x17d5 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6990 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 08:03:45 (1740488625) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00264166 s, 3.8 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.26366 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 08:03:52 (1740488632) 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 3 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 4 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 5 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 9 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/intel_rapl/refcnt': 'No such device' Iteration 12 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 16 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/acpi_cpufreq/refcnt': 'No such device' ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 28 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 29 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 31 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 32 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 36 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 50 Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 192.168.201.153@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.201.153@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions oleg153-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre2 PASS 109 (444s) == sanityn test 110: do not grant another lock on resend ========================================================== 08:11:16 (1740489076) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-02-25 08:11:16.000000000 -0500 Modify: 2025-02-25 08:11:16.000000000 -0500 Change: 2025-02-25 08:11:16.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg153-client.virtnet oleg153-client.virtnet PASS 110 (65s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 08:12:21 (1740489141) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 08:12:22 (1740489142) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 08:12:23 (1740489143) lustre-OST0000-osc-ffff8800b58db800 active. lustre-OST0001-osc-ffff8800b58db800 active. lustre-OST0000-osc-ffff8800b58db800 active. lustre-OST0001-osc-ffff8800b58db800 active. MGC192.168.201.153@tcp active. PASS 113 (3s) == sanityn test 114: implicit default LMV inherit ======== 08:12:26 (1740489146) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 08:12:27 (1740489147) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (2s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 08:12:29 (1740489149) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 08:12:30 (1740489150) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=20 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.0373 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 20.0459 s, 0.2 kB/s , 20.0402 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 20.0607 s, 0.2 kB/s , 20.0523 s, 0.2 kB/s , 20.0469 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.0502 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1261 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1396 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.0533 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1031 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.1478 s, 0.1 kB/s , 40.1273 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.1604 s, 0.1 kB/s , 40.1131 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1422 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.194 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2354 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.1543 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.1732 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2533 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2114 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2863 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 60.2196 s, 0.1 kB/s , 60.2664 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3338 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3023 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.27 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2781 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3135 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.397 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.333 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3801 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3904 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3933 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3972 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.427 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.392 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.441 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.472 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg153-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58db800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b58db800.ost_server_uuid in FULL state after 0 sec PASS 200 (107s) cleanup: ====================================================== == sanityn test complete, duration 4461 sec ============== 08:14:18 (1740489258) === sanityn: start cleanup 08:14:18 (1740489258) === Stopping clients: oleg153-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg153-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 08:15:23 (1740489323) ===