-----============= acceptance-small: sanityn ============----- Wed Oct 22 04:16:14 EDT 2025 oleg143-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Started clients oleg143-client.virtnet: 192.168.201.143@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg143-client.virtnet: executing check_config_client /mnt/lustre oleg143-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg143-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffffa0fe830c7800.idle_timeout=debug osc.lustre-OST0000-osc-ffffa0fe8801e800.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0fe830c7800.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0fe8801e800.idle_timeout=debug disable quota as required oleg143-server: oleg143-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109242 s, 96.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7118] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 04:16:28 (1761120988) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 04:16:30 (1761120990) -rw-r--r-- 1 root root 0 Oct 22 04:16 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 04:16:33 (1761120993) -rw-r--r-- 1 root root 0 Oct 22 04:16 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 04:16:35 (1761120995) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 04:16:37 (1761120997) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 04:16:39 (1761120999) total 1 -rwxrwxrwx 1 root root 0 Oct 22 04:16 f2a -rwxrwxrwx 1 root root 0 Oct 22 04:16 f2b total 1 -rwxrwxrwx 1 root root 0 Oct 22 04:16 f2a -rwxrwxrwx 1 root root 0 Oct 22 04:16 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00272578 s, 188 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 04:16:41 (1761121001) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 04:16:42 (1761121002) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0835694 s, 25.1 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 04:16:44 (1761121004) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 04:16:46 (1761121006) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 04:16:49 (1761121009) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 04:16:51 (1761121011) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 04:16:53 (1761121013) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 04:16:56 (1761121016) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 04:16:58 (1761121018) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 04:17:00 (1761121020) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165379 s, 63.4 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0117517 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 04:17:02 (1761121022) 1+0 records in 1+0 records out 1 byte copied, 0.00184822 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00209319 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00145116 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00136454 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00175835 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00166296 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00181163 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0017604 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00150813 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.001567 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00165323 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00153058 s, 0.7 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 04:17:04 (1761121024) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00352823 s, 871 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00832897 s, 492 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000244492 s, 12.6 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 04:17:06 (1761121026) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 04:17:08 (1761121028) warning: '-runtime' deprecated, use '-t runtime' instead - link 2267 (time 1761121039.50 total 10.00 last 226.67) - link 4366 (time 1761121049.50 total 20.00 last 209.89) - link 6362 (time 1761121059.50 total 30.00 last 199.58) - link 8217 (time 1761121069.50 total 40.00 last 185.49) - link 10000 (time 1761121077.81 total 48.32 last 214.50) - link 12119 (time 1761121087.81 total 58.32 last 211.83) - link 14267 (time 1761121097.82 total 68.32 last 214.74) - link 16390 (time 1761121107.82 total 78.32 last 212.28) - link 18567 (time 1761121117.82 total 88.33 last 217.59) - link 20000 (time 1761121124.14 total 94.64 last 226.96) total: 21116 link in 100.00 seconds: 211.15 ops/second using seed 4135845065 running for 100 seconds - stat 10000 (time 1761121034 ; total 4 ; last 4) - stat 20000 (time 1761121038 ; total 8 ; last 4) - stat 30000 (time 1761121042 ; total 12 ; last 4) - stat 40000 (time 1761121047 ; total 17 ; last 5) - stat 50000 (time 1761121051 ; total 21 ; last 4) - stat 60000 (time 1761121056 ; total 26 ; last 5) - stat 70000 (time 1761121060 ; total 30 ; last 4) - stat 80000 (time 1761121065 ; total 35 ; last 5) - stat 90000 (time 1761121070 ; total 40 ; last 5) - stat 100000 (time 1761121074 ; total 44 ; last 4) - stat 110000 (time 1761121079 ; total 49 ; last 5) - stat 120000 (time 1761121088 ; total 58 ; last 9) - stat 130000 (time 1761121098 ; total 68 ; last 10) - stat 140000 (time 1761121108 ; total 78 ; last 10) - stat 149994 (time 1761121119 ; total 89 ; last 11) - stat 150000 (time 1761121119 ; total 89 ; last 0) - stat 160000 (time 1761121129 ; total 99 ; last 10) total: 161318 stats in 100 seconds: 1613.180054 stats/second - unlinked 0 (time 1761121080 ; total 0 ; last 0) - unlinked 10000 (time 1761121137 ; total 57 ; last 57) - unlinked 20000 (time 1761121164 ; total 84 ; last 27) unlink(/mnt/lustre2/lockdir/lockfile21116) error: No such file or directory total: 21115 unlinks in 87 seconds: 242.701157 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19673) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19676) - No such process PASS 12 (141s) == sanityn test 13: test directory page revocation ======= 04:19:29 (1761121169) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 04:19:31 (1761121171) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20845 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 04:19:33 (1761121173) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21417 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 04:19:35 (1761121175) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 21988 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 04:19:36 (1761121176) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22564 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 04:19:38 (1761121178) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23139 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 04:19:41 (1761121181) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7520256 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7520256kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 04:19:52 (1761121192) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0715482 s, 29.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0528888 s, 39.7 MB/s Chance of close/open is 1 in 50 Seed set to 1193 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1761121193.497879 mapwrite 0x338e1a thru 0x3414d2 (0x86b9 bytes) 000003[0] 1761121193.542227 write 0x6f9692 thru 0x70500d (0xb97c bytes) 000004[0] 1761121193.544973 write 0x5fa2e5 thru 0x60648a (0xc1a6 bytes) 000006[1] 1761121193.556463 read 0x008799 thru 0x0145d5 (0xbe3d bytes) truncating to largest ever: 0x2161b4 000009[1] 1761121193.567859 trunc from 0x70500e to 0x2161b3 (0x4eee5a bytes) 000011[1] 1761121193.579024 mapread 0x158985 thru 0x1640b7 (0xb733 bytes) 000012[0] 1761121193.586205 write 0x7a2c94 thru 0x7a9ded (0x715a bytes) 000013[0] 1761121193.595253 mapwrite 0x949c36 thru 0x953bdc (0x9fa7 bytes) 000015[0] 1761121193.622294 trunc from 0x953bdd to 0x0df3c0 (0x87481c bytes) 000020[0] 1761121193.638894 mapread 0x0d4d3c thru 0x0dc8bc (0x7b81 bytes) 000022[1] 1761121193.651921 mapwrite 0x8b783e thru 0x8bf56f (0x7d32 bytes) 000026[0] 1761121193.690840 mapwrite 0x17382a thru 0x1765af (0x2d86 bytes) 000027[1] 1761121193.712912 write 0x684f21 thru 0x68ee43 (0x9f23 bytes) truncating to largest ever: 0x6901ec 000028[1] 1761121193.720304 trunc from 0x8bf570 to 0x6901eb (0x22f384 bytes) 000029[0] 1761121193.733003 trunc from 0x6901ec to 0x65243b (0x3ddb0 bytes) 000030[1] 1761121193.742051 mapread 0x1bedac thru 0x1ca11c (0xb371 bytes) 000032[0] 1761121193.763121 read 0x164d92 thru 0x166284 (0x14f3 bytes) 000035[1] 1761121193.774842 write 0x42de5e thru 0x430f35 (0x30d8 bytes) 000036[0] 1761121193.783601 mapread 0x2e7a9e thru 0x2f3724 (0xbc87 bytes) 000037[1] 1761121193.797943 read 0x5fcb6d thru 0x5ffb2d (0x2fc1 bytes) 000040[1] 1761121193.800090 mapread 0x287950 thru 0x28dad0 (0x6181 bytes) 000041[1] 1761121193.813702 write 0x4376a7 thru 0x44714f (0xfaa9 bytes) 000042[1] 1761121193.823648 mapread 0x2d6cc6 thru 0x2e2837 (0xbb72 bytes) 000044[1] 1761121193.849328 trunc from 0x65243c to 0x5edbf2 (0x64849 bytes) 000047[0] 1761121193.865114 trunc from 0x5edbf3 to 0x146f39 (0x4a6cb9 bytes) 000048[1] 1761121193.879566 mapread 0x11482c thru 0x114dd9 (0x5ae bytes) 000051[1] 1761121193.894588 mapread 0x11f8d4 thru 0x129f98 (0xa6c5 bytes) 000054[1] 1761121193.911802 trunc from 0x146f3a to 0x14cc57 (0x5d1e bytes) 000059[1] 1761121193.929230 read 0x10e839 thru 0x116120 (0x78e8 bytes) 000060[1] 1761121193.932463 mapwrite 0x5ad6e3 thru 0x5ba23c (0xcb5a bytes) 000063[0] 1761121193.989654 read 0x50718a thru 0x510bb2 (0x9a29 bytes) 000064[0] 1761121194.003606 mapwrite 0x9a9a1d thru 0x9b22ac (0x8890 bytes) 000066[1] 1761121194.061952 mapread 0x37354d thru 0x37796a (0x441e bytes) 000067[1] 1761121194.075489 mapread 0x3a1006 thru 0x3a2312 (0x130d bytes) 000071[1] 1761121194.084621 read 0x350f6f thru 0x35bd76 (0xae08 bytes) 000072[1] 1761121194.094572 read 0x568f07 thru 0x572445 (0x953f bytes) truncating to largest ever: 0x99ecc4 000075[0] 1761121194.100225 trunc from 0x9b22ad to 0x99ecc3 (0x135e9 bytes) 000076[0] 1761121194.109355 read 0x271957 thru 0x27bb9e (0xa248 bytes) 000078[0] 1761121194.147839 trunc from 0x99ecc4 to 0x217f47 (0x786d7c bytes) 000079[0] 1761121194.162352 mapwrite 0x7a0b75 thru 0x7ae74a (0xdbd6 bytes) 000080[0] 1761121194.206378 mapwrite 0x317c4a thru 0x321dd4 (0xa18b bytes) 000081[0] 1761121194.238930 mapwrite 0x36d01f thru 0x3739dc (0x69be bytes) 000082[0] 1761121194.260958 trunc from 0x7ae74b to 0x5d687b (0x1d7ecf bytes) 000084[1] 1761121194.273677 trunc from 0x5d687c to 0x86e83d (0x297fc2 bytes) 000085[1] 1761121194.284792 trunc from 0x86e83e to 0x0ebbbd (0x782c80 bytes) 000087[0] 1761121194.294735 write 0x989e66 thru 0x98a176 (0x311 bytes) 000089[0] 1761121194.306958 mapread 0x407613 thru 0x416240 (0xec2e bytes) 000093[0] 1761121194.345486 read 0x06e0f5 thru 0x07b20b (0xd117 bytes) 000094[0] 1761121194.382062 mapread 0x22648e thru 0x2323f9 (0xbf6c bytes) 000095[1] 1761121194.382579 write 0x47d73f thru 0x48b9bf (0xe281 bytes) 000096[0] 1761121194.396553 trunc from 0x98a177 to 0x461ce1 (0x528495 bytes) 000097[1] 1761121194.412446 trunc from 0x461ce2 to 0x60132f (0x19f64e bytes) 000100[0] 1761121194.433700 read 0x20c60c thru 0x21bc48 (0xf63d bytes) 000101[1] 1761121194.448402 mapread 0x19afb1 thru 0x19f69a (0x46ea bytes) 000102[1] 1761121194.461343 mapread 0x5da34c thru 0x5dc2e0 (0x1f95 bytes) 000104[0] 1761121194.469719 mapwrite 0x91bac9 thru 0x91f98d (0x3ec5 bytes) 000105[1] 1761121194.531167 mapwrite 0x46d967 thru 0x47a9bb (0xd055 bytes) 000106[0] 1761121194.585295 mapwrite 0x68555e thru 0x68ecd5 (0x9778 bytes) 000107[0] 1761121194.618479 trunc from 0x91f98e to 0x63f560 (0x2e042d bytes) 000109[0] 1761121194.633656 write 0x41404f thru 0x423039 (0xefeb bytes) 000111[1] 1761121194.642556 read 0x329891 thru 0x329c1c (0x38c bytes) 000112[1] 1761121194.654703 read 0x5ce51a thru 0x5dbd4a (0xd831 bytes) 000113[1] 1761121194.661973 read 0x18c43f thru 0x194fc7 (0x8b89 bytes) 000114[0] 1761121194.670555 mapread 0x19aae4 thru 0x1a21e9 (0x7706 bytes) 000115[1] 1761121194.682795 mapwrite 0x78d6d2 thru 0x79c220 (0xeb4f bytes) 000116[1] 1761121194.749377 mapwrite 0x6abf70 thru 0x6b5040 (0x90d1 bytes) 000118[0] 1761121194.769864 mapread 0x3bf796 thru 0x3c7615 (0x7e80 bytes) 000119[0] 1761121194.794579 mapread 0x4bc059 thru 0x4c52f2 (0x929a bytes) 000123[1] 1761121194.815990 trunc from 0x79c221 to 0x03bc0f (0x760611 bytes) 000127[1] 1761121194.835612 read 0x001cb7 thru 0x002b66 (0xeb0 bytes) 000128[0] 1761121194.842968 read 0x039008 thru 0x03bc0f (0x2c08 bytes) 000129[1] 1761121194.850309 mapwrite 0x1a6830 thru 0x1a7b14 (0x12e5 bytes) 000130[1] 1761121194.880343 trunc from 0x1a7b15 to 0x7c3cd3 (0x61c1bf bytes) 000132[1] 1761121194.890935 mapwrite 0x1ed5d4 thru 0x1f9516 (0xbf43 bytes) 000135[1] 1761121194.921576 write 0x8b79ac thru 0x8bce30 (0x5485 bytes) 000136[1] 1761121194.926902 mapread 0x75d31a thru 0x76436e (0x7055 bytes) 000137[0] 1761121194.936502 mapread 0x1867a7 thru 0x193992 (0xd1ec bytes) 000139[1] 1761121194.967707 trunc from 0x8bce31 to 0x98f679 (0xd2849 bytes) 000142[1] 1761121194.980285 write 0x4eb19c thru 0x4f3471 (0x82d6 bytes) 000143[0] 1761121194.987124 mapwrite 0x4fc15b thru 0x50620c (0xa0b2 bytes) 000144[1] 1761121195.070077 mapread 0x051c8f thru 0x053874 (0x1be6 bytes) 000148[1] 1761121195.094376 write 0x977c1f thru 0x979c73 (0x2055 bytes) 000149[1] 1761121195.103619 mapread 0x925ca1 thru 0x928325 (0x2685 bytes) 000153[1] 1761121195.114168 read 0x5ea3ed thru 0x5f13e5 (0x6ff9 bytes) 000155[0] 1761121195.145167 trunc from 0x98f67a to 0x596abe (0x3f8bbb bytes) 000157[0] 1761121195.168523 read 0x585a07 thru 0x58d3e4 (0x79de bytes) 000159[1] 1761121195.172181 mapread 0x4b397d thru 0x4bdff6 (0xa67a bytes) 000160[1] 1761121195.194732 write 0x55e34e thru 0x569639 (0xb2ec bytes) 000162[0] 1761121195.200055 mapread 0x00d667 thru 0x01b335 (0xdccf bytes) 000164[0] 1761121195.226625 mapwrite 0x7a28d3 thru 0x7a7c8b (0x53b9 bytes) 000165[0] 1761121195.274895 mapwrite 0x59910a thru 0x5a691e (0xd815 bytes) 000166[0] 1761121195.290808 read 0x21f5b8 thru 0x2245af (0x4ff8 bytes) 000169[1] 1761121195.318691 mapread 0x1a9b70 thru 0x1b337a (0x980b bytes) 000171[0] 1761121195.348672 mapread 0x12ccf6 thru 0x1301b3 (0x34be bytes) 000172[1] 1761121195.360943 mapread 0x3b7073 thru 0x3ba4c5 (0x3453 bytes) 000173[1] 1761121195.372403 read 0x02e3d5 thru 0x038deb (0xaa17 bytes) 000175[0] 1761121195.414392 mapread 0x4b446a thru 0x4bd794 (0x932b bytes) 000178[1] 1761121195.429669 mapread 0x2f45ca thru 0x302c01 (0xe638 bytes) 000179[1] 1761121195.430224 read 0x491955 thru 0x494883 (0x2f2f bytes) 000180[0] 1761121195.430550 read 0x5c5066 thru 0x5c9391 (0x432c bytes) 000181[0] 1761121195.443302 write 0x448014 thru 0x452102 (0xa0ef bytes) 000182[1] 1761121195.459849 mapwrite 0x3a078e thru 0x3a1caf (0x1522 bytes) 000183[0] 1761121195.500594 read 0x13d8fd thru 0x143532 (0x5c36 bytes) 000184[1] 1761121195.538090 write 0x05ec13 thru 0x066819 (0x7c07 bytes) 000186[1] 1761121195.553774 trunc from 0x7a7c8c to 0x2867ca (0x5214c1 bytes) 000187[1] 1761121195.567188 mapread 0x2810c7 thru 0x2867ca (0x5704 bytes) 000189[1] 1761121195.581359 trunc from 0x2867cb to 0x62409e (0x39d8d4 bytes) 000192[1] 1761121195.593214 write 0x853edd thru 0x8576a3 (0x37c7 bytes) 000193[0] 1761121195.594265 mapwrite 0x1ac799 thru 0x1ae1c3 (0x1a2b bytes) 000195[0] 1761121195.623880 write 0x54364d thru 0x55269a (0xf04e bytes) 000197[1] 1761121195.635787 mapwrite 0x723b59 thru 0x725cd0 (0x2178 bytes) 000199[1] 1761121195.663342 mapread 0x127676 thru 0x136d69 (0xf6f4 bytes) 000200[1] 1761121195.682626 mapwrite 0x267499 thru 0x26c192 (0x4cfa bytes) 000201[0] 1761121195.705398 mapread 0x710890 thru 0x7175e8 (0x6d59 bytes) 000208[0] 1761121195.739943 mapread 0x0cb137 thru 0x0d5cbc (0xab86 bytes) 000209[0] 1761121195.762718 mapwrite 0x6a882b thru 0x6b18c4 (0x909a bytes) 000210[1] 1761121195.792333 trunc from 0x8576a4 to 0x656803 (0x200ea0 bytes) 000212[0] 1761121195.810927 write 0x7981cd thru 0x79c173 (0x3fa7 bytes) 000213[0] 1761121195.818209 trunc from 0x79c174 to 0x4aabd9 (0x2f159a bytes) 000215[1] 1761121195.830245 trunc from 0x4aabda to 0x748b68 (0x29df8f bytes) 000216[1] 1761121195.839154 read 0x4a5fa9 thru 0x4b014f (0xa1a7 bytes) 000217[0] 1761121195.859734 mapwrite 0x45e6ad thru 0x46852e (0x9e82 bytes) 000219[0] 1761121195.901734 mapwrite 0x5c9ec2 thru 0x5d8f49 (0xf088 bytes) 000220[0] 1761121195.948467 mapwrite 0x375a76 thru 0x380b87 (0xb112 bytes) 000221[0] 1761121195.980902 trunc from 0x748b69 to 0x299f20 (0x4aec48 bytes) 000223[0] 1761121195.991619 read 0x09b536 thru 0x0a9ccf (0xe79a bytes) 000228[1] 1761121196.021535 trunc from 0x299f21 to 0x4dafd7 (0x2410b7 bytes) 000231[0] 1761121196.038256 mapread 0x0d7719 thru 0x0e2d29 (0xb611 bytes) 000234[0] 1761121196.069580 mapread 0x2fe32e thru 0x304e55 (0x6b28 bytes) 000235[1] 1761121196.080660 write 0x6ba624 thru 0x6bfb5a (0x5537 bytes) 000237[1] 1761121196.088499 mapread 0x5024b9 thru 0x50b213 (0x8d5b bytes) 000238[0] 1761121196.105649 mapread 0x1aef0c thru 0x1bb0a0 (0xc195 bytes) 000240[0] 1761121196.129649 mapwrite 0x190f4a thru 0x194897 (0x394e bytes) 000241[1] 1761121196.156948 write 0x70f71a thru 0x7102c8 (0xbaf bytes) 000242[1] 1761121196.159511 trunc from 0x7102c9 to 0x4b19ec (0x25e8dc bytes) 000243[1] 1761121196.168399 read 0x174505 thru 0x17b314 (0x6e10 bytes) 000244[1] 1761121196.187905 read 0x126683 thru 0x13538d (0xed0b bytes) 000245[0] 1761121196.191788 write 0x32877b thru 0x334f17 (0xc79d bytes) 000246[1] 1761121196.202795 mapread 0x2517c5 thru 0x25a38e (0x8bca bytes) 000247[0] 1761121196.208639 write 0x356755 thru 0x363958 (0xd204 bytes) 000248[0] 1761121196.215825 mapwrite 0x5f641a thru 0x601022 (0xac09 bytes) 000249[0] 1761121196.261869 read 0x15ba76 thru 0x15e270 (0x27fb bytes) 000250[0] 1761121196.292366 mapread 0x38bb33 thru 0x3961a6 (0xa674 bytes) 000252[1] 1761121196.293073 mapread 0x539fe1 thru 0x53c779 (0x2799 bytes) 000253[1] 1761121196.308512 read 0x32e455 thru 0x32f124 (0xcd0 bytes) 000254[0] 1761121196.318541 mapread 0x0ba36d thru 0x0c6917 (0xc5ab bytes) 000257[1] 1761121196.344074 trunc from 0x601023 to 0x607bfc (0x6bda bytes) 000258[1] 1761121196.361305 read 0x58cff9 thru 0x59a793 (0xd79b bytes) 000260[0] 1761121196.374786 mapwrite 0x2c70c2 thru 0x2ce337 (0x7276 bytes) 000264[0] 1761121196.418907 read 0x3c8f9f thru 0x3d4a01 (0xba63 bytes) 000266[0] 1761121196.445104 write 0x8def5a thru 0x8e99ec (0xaa93 bytes) 000268[0] 1761121196.450040 read 0x80200c thru 0x809e8e (0x7e83 bytes) 000269[0] 1761121196.455808 write 0x1cd672 thru 0x1d3b35 (0x64c4 bytes) 000270[1] 1761121196.466046 mapwrite 0x1ecfe6 thru 0x1f1cbd (0x4cd8 bytes) 000272[0] 1761121196.501468 trunc from 0x8e99ed to 0x78aaa9 (0x15ef43 bytes) 000273[0] 1761121196.509431 trunc from 0x78aaaa to 0x720597 (0x6a512 bytes) 000274[1] 1761121196.513132 mapwrite 0x0fc5e3 thru 0x10074b (0x4169 bytes) 000277[0] 1761121196.556007 mapwrite 0x3e9536 thru 0x3eb082 (0x1b4d bytes) 000278[0] 1761121196.579463 mapwrite 0x06a623 thru 0x075386 (0xad64 bytes) 000279[0] 1761121196.617213 trunc from 0x720598 to 0x8587cb (0x138234 bytes) 000280[1] 1761121196.625504 mapwrite 0x123ca6 thru 0x130604 (0xc95f bytes) 000281[0] 1761121196.677599 read 0x749510 thru 0x74dc85 (0x4776 bytes) 000282[0] 1761121196.695015 read 0x218427 thru 0x219e2c (0x1a06 bytes) 000283[1] 1761121196.726043 trunc from 0x8587cc to 0x4a83e6 (0x3b03e5 bytes) 000284[0] 1761121196.742090 mapread 0x3fb25f thru 0x404d46 (0x9ae8 bytes) 000285[1] 1761121196.762200 trunc from 0x4a83e7 to 0x08efd6 (0x419410 bytes) 000287[1] 1761121196.772086 mapwrite 0x6dd721 thru 0x6e9b6c (0xc44c bytes) 000289[1] 1761121196.818891 read 0x07f07b thru 0x085502 (0x6488 bytes) 000290[1] 1761121196.856743 trunc from 0x6e9b6d to 0x932317 (0x2487ab bytes) 000291[1] 1761121196.862563 write 0x6dd65c thru 0x6e11f5 (0x3b9a bytes) 000292[1] 1761121196.863083 read 0x3c0dba thru 0x3c22b6 (0x14fd bytes) 000293[1] 1761121196.870532 trunc from 0x932318 to 0x66a924 (0x2c79f3 bytes) 000295[0] 1761121196.890087 mapwrite 0x2fcd24 thru 0x2fe6b8 (0x1995 bytes) 000296[1] 1761121196.918906 read 0x467594 thru 0x468258 (0xcc5 bytes) 000297[1] 1761121196.933899 mapread 0x001edd thru 0x002cc5 (0xde9 bytes) 000299[0] 1761121196.940917 trunc from 0x66a925 to 0x745e7c (0xdb558 bytes) 000302[0] 1761121196.964205 write 0x88e034 thru 0x88ed9d (0xd6a bytes) 000304[1] 1761121196.972711 write 0x3499aa thru 0x35034d (0x69a4 bytes) 000306[1] 1761121196.985594 write 0x6917e6 thru 0x697705 (0x5f20 bytes) 000307[1] 1761121197.004899 mapwrite 0x915bd0 thru 0x91dc6c (0x809d bytes) 000308[0] 1761121197.029822 read 0x242858 thru 0x246829 (0x3fd2 bytes) 000309[1] 1761121197.047014 read 0x765a13 thru 0x766455 (0xa43 bytes) 000311[1] 1761121197.048658 read 0x3ac8a1 thru 0x3ae98d (0x20ed bytes) 000312[1] 1761121197.076629 mapwrite 0x9e4ec7 thru 0x9e9812 (0x494c bytes) 000313[1] 1761121197.110175 trunc from 0x9e9813 to 0x3cdb3b (0x61bcd7 bytes) 000315[0] 1761121197.122913 write 0x7981dd thru 0x79c24b (0x406f bytes) 000316[1] 1761121197.130639 mapread 0x60f224 thru 0x6141a8 (0x4f85 bytes) 000317[0] 1761121197.158067 write 0x6ceacf thru 0x6de615 (0xfb47 bytes) 000319[1] 1761121197.190951 write 0x5a22f4 thru 0x5aa52e (0x823b bytes) 000321[0] 1761121197.207575 mapread 0x4ecf3b thru 0x4f24c4 (0x558a bytes) 000323[1] 1761121197.217741 trunc from 0x79c24c to 0x10caf9 (0x68f752 bytes) 000325[0] 1761121197.233744 mapread 0x0090f0 thru 0x00e6e6 (0x55f7 bytes) 000326[1] 1761121197.246289 mapwrite 0x3d47f6 thru 0x3e07ab (0xbfb6 bytes) 000328[0] 1761121197.291051 write 0x5b7cb0 thru 0x5c1d44 (0xa095 bytes) 000329[1] 1761121197.297991 trunc from 0x5c1d45 to 0x65bcdd (0x99f99 bytes) 000331[1] 1761121197.312543 trunc from 0x65bcde to 0x37dd2a (0x2ddfb3 bytes) 000332[1] 1761121197.317237 read 0x1c3b9d thru 0x1d0dc2 (0xd226 bytes) 000333[0] 1761121197.332200 mapread 0x2e002e thru 0x2ee6a4 (0xe677 bytes) 000334[1] 1761121197.358472 write 0x9b4859 thru 0x9b4992 (0x13a bytes) 000336[0] 1761121197.365817 trunc from 0x9b4993 to 0x598118 (0x41c87a bytes) 000337[0] 1761121197.380940 read 0x369e79 thru 0x374aa5 (0xac2d bytes) 000338[0] 1761121197.399286 mapread 0x24bac4 thru 0x252d42 (0x727f bytes) 000340[0] 1761121197.410282 mapwrite 0x54216d thru 0x54f95e (0xd7f2 bytes) 000341[0] 1761121197.426034 mapwrite 0x55ba8f thru 0x55c76f (0xce1 bytes) 000342[1] 1761121197.442454 trunc from 0x598119 to 0x330a84 (0x267694 bytes) 000345[0] 1761121197.454473 mapread 0x120cac thru 0x1295cd (0x8922 bytes) 000348[0] 1761121197.483373 trunc from 0x330a85 to 0x4c6802 (0x195d7e bytes) 000350[0] 1761121197.497321 mapwrite 0x6fcfa7 thru 0x702882 (0x58dc bytes) 000354[1] 1761121197.534462 mapwrite 0x7cc7ce thru 0x7d39d6 (0x7209 bytes) 000355[0] 1761121197.590692 trunc from 0x7d39d7 to 0x177438 (0x65c59e bytes) 000357[1] 1761121197.608795 read 0x02bfd6 thru 0x03796a (0xb995 bytes) 000358[0] 1761121197.622078 mapwrite 0x61e693 thru 0x62d8c5 (0xf233 bytes) 000361[0] 1761121197.669208 write 0x5fc2ec thru 0x5fd1a0 (0xeb5 bytes) 000368[0] 1761121197.687479 mapwrite 0x988109 thru 0x98ac8a (0x2b82 bytes) 000369[1] 1761121197.713841 write 0x103bb4 thru 0x110a67 (0xceb4 bytes) 000370[1] 1761121197.728944 mapread 0x3dcc84 thru 0x3df93e (0x2cbb bytes) 000372[1] 1761121197.744722 mapwrite 0x07ae8a thru 0x07c2f1 (0x1468 bytes) 000373[0] 1761121197.812142 mapread 0x643917 thru 0x647a84 (0x416e bytes) 000375[1] 1761121197.833808 write 0x25fc99 thru 0x267a79 (0x7de1 bytes) 000378[0] 1761121197.844173 mapwrite 0x95676e thru 0x957d18 (0x15ab bytes) 000379[0] 1761121197.873593 read 0x106ca3 thru 0x1135be (0xc91c bytes) 000381[1] 1761121197.889306 write 0x6b02b8 thru 0x6b808e (0x7dd7 bytes) 000382[0] 1761121197.896789 trunc from 0x98ac8b to 0x6f6709 (0x294581 bytes) 000385[0] 1761121197.915658 trunc from 0x6f670a to 0x840543 (0x149e3a bytes) 000386[1] 1761121197.920485 mapread 0x6e15e0 thru 0x6e65dc (0x4ffd bytes) 000387[1] 1761121197.932824 read 0x22cdba thru 0x23b93e (0xeb85 bytes) 000389[1] 1761121197.940022 mapwrite 0x5c04b9 thru 0x5cb17d (0xacc5 bytes) 000390[0] 1761121197.997214 trunc from 0x840544 to 0x7b8967 (0x87bdc bytes) 000391[0] 1761121198.011664 read 0x7055b3 thru 0x70992c (0x437a bytes) 000392[0] 1761121198.023278 read 0x40d900 thru 0x41a445 (0xcb46 bytes) 000393[0] 1761121198.051349 read 0x72ddf4 thru 0x731108 (0x3315 bytes) 000394[0] 1761121198.051935 mapread 0x11754c thru 0x122272 (0xad27 bytes) 000395[0] 1761121198.069114 trunc from 0x7b8968 to 0x294e00 (0x523b67 bytes) 000397[1] 1761121198.087688 mapread 0x2708b8 thru 0x2774d9 (0x6c22 bytes) 000398[1] 1761121198.104974 trunc from 0x294e01 to 0x71c796 (0x487996 bytes) 000399[0] 1761121198.115512 read 0x192226 thru 0x19f450 (0xd22b bytes) 000400[0] 1761121198.123994 mapread 0x439b74 thru 0x43c225 (0x26b2 bytes) 000401[1] 1761121198.182049 read 0x2f8d7a thru 0x302c31 (0x9eb8 bytes) 000403[1] 1761121198.225496 write 0x31ee30 thru 0x32104b (0x221c bytes) 000405[1] 1761121198.245332 write 0x9f9d82 thru 0x9fffff (0x627e bytes) 000406[0] 1761121198.250647 mapread 0x1a94c9 thru 0x1ad979 (0x44b1 bytes) 000408[0] 1761121198.290661 trunc from 0xa00000 to 0x26ca9d (0x793562 bytes) 000412[0] 1761121198.317528 write 0x402eac thru 0x403327 (0x47c bytes) 000413[1] 1761121198.323042 mapread 0x1725cb thru 0x178e5f (0x6895 bytes) 000414[0] 1761121198.337789 mapwrite 0x33f72c thru 0x3420d5 (0x29aa bytes) 000418[1] 1761121198.364618 mapwrite 0x36ecf1 thru 0x37287c (0x3b8c bytes) 000420[0] 1761121198.421620 mapread 0x0b90fa thru 0x0c163e (0x8545 bytes) 000421[0] 1761121198.439617 read 0x190a8f thru 0x198d2a (0x829c bytes) 000422[0] 1761121198.466094 write 0x2e37fa thru 0x2f04fd (0xcd04 bytes) 000423[1] 1761121198.467356 trunc from 0x403328 to 0x2c6cbc (0x13c66b bytes) 000425[1] 1761121198.483190 read 0x07186f thru 0x07b58f (0x9d21 bytes) 000426[0] 1761121198.498743 read 0x0c217b thru 0x0cf61b (0xd4a1 bytes) 000427[1] 1761121198.513219 read 0x294e75 thru 0x2a34d6 (0xe662 bytes) 000428[0] 1761121198.522152 mapread 0x1bc28b thru 0x1c7719 (0xb48f bytes) 000429[0] 1761121198.542511 mapread 0x047206 thru 0x04e869 (0x7664 bytes) 000431[1] 1761121198.555524 read 0x280dfc thru 0x28cad2 (0xbcd7 bytes) 000433[0] 1761121198.561523 trunc from 0x2c6cbd to 0x7533d0 (0x48c714 bytes) 000434[0] 1761121198.576687 trunc from 0x7533d1 to 0x40bcf8 (0x3476d8 bytes) 000435[0] 1761121198.586319 trunc from 0x40bcf9 to 0x19b326 (0x2709d2 bytes) 000436[1] 1761121198.591695 write 0x887416 thru 0x88de67 (0x6a52 bytes) 000437[1] 1761121198.599934 mapwrite 0x616ba2 thru 0x61a9d8 (0x3e37 bytes) 000438[0] 1761121198.632727 mapwrite 0x0805fd thru 0x089452 (0x8e56 bytes) 000439[0] 1761121198.670547 trunc from 0x88de68 to 0x6b2e3a (0x1db02d bytes) 000440[1] 1761121198.680199 mapwrite 0x68eacd thru 0x68fb58 (0x108c bytes) 000442[0] 1761121198.710128 trunc from 0x6b2e3b to 0x2918d1 (0x421569 bytes) 000445[0] 1761121198.724121 mapread 0x1f382a thru 0x1fd2f0 (0x9ac7 bytes) 000446[1] 1761121198.743607 trunc from 0x2918d2 to 0x3437cb (0xb1efa bytes) 000447[1] 1761121198.757767 mapwrite 0x6ffdbb thru 0x70a0bb (0xa301 bytes) 000451[1] 1761121198.827670 mapwrite 0x816feb thru 0x81d94a (0x6960 bytes) 000452[0] 1761121198.867334 mapread 0x0a41af thru 0x0adff4 (0x9e46 bytes) 000456[1] 1761121198.899138 write 0x93d5fc thru 0x948b0d (0xb512 bytes) 000457[1] 1761121198.900128 write 0x29df38 thru 0x2a9e1e (0xbee7 bytes) 000458[1] 1761121198.906176 mapwrite 0x3bfeaa thru 0x3c3972 (0x3ac9 bytes) 000459[1] 1761121198.928849 read 0x5caf78 thru 0x5d1ec6 (0x6f4f bytes) 000460[0] 1761121198.934028 write 0x96727d thru 0x967ea0 (0xc24 bytes) 000463[0] 1761121198.973615 mapread 0x7f8556 thru 0x7f9199 (0xc44 bytes) 000464[0] 1761121198.982947 read 0x6f8611 thru 0x707868 (0xf258 bytes) 000465[0] 1761121199.017285 trunc from 0x967ea1 to 0x76f876 (0x1f862a bytes) 000466[1] 1761121199.032348 write 0x5b75c3 thru 0x5c66e0 (0xf11e bytes) 000469[0] 1761121199.050613 trunc from 0x76f877 to 0x5d3572 (0x19c304 bytes) 000471[1] 1761121199.067956 trunc from 0x5d3573 to 0x7d17fa (0x1fe288 bytes) 000473[1] 1761121199.078404 trunc from 0x7d17fb to 0x4e5d00 (0x2ebafa bytes) 000475[0] 1761121199.091698 read 0x0e3e34 thru 0x0f19e3 (0xdbb0 bytes) 000478[1] 1761121199.103466 write 0x1ddf53 thru 0x1e70db (0x9189 bytes) 000480[1] 1761121199.115102 write 0x19524a thru 0x1a1ae0 (0xc897 bytes) 000481[0] 1761121199.122782 read 0x1baf3f thru 0x1c71dc (0xc29e bytes) 000482[0] 1761121199.173063 mapread 0x3f12f1 thru 0x3f5db6 (0x4ac6 bytes) 000483[1] 1761121199.173541 write 0x546054 thru 0x54f61a (0x95c7 bytes) 000484[0] 1761121199.178681 mapread 0x08d0c0 thru 0x09979a (0xc6db bytes) 000485[0] 1761121199.197538 mapwrite 0x3e9177 thru 0x3f7e98 (0xed22 bytes) 000486[0] 1761121199.242938 write 0x22f743 thru 0x23138a (0x1c48 bytes) 000488[0] 1761121199.259325 write 0x88445e thru 0x888cf7 (0x489a bytes) 000489[0] 1761121199.260523 read 0x54667f thru 0x553e13 (0xd795 bytes) 000490[0] 1761121199.296080 write 0x6ed8cb thru 0x6f4194 (0x68ca bytes) 000493[0] 1761121199.309936 mapwrite 0x436eb0 thru 0x43d70e (0x685f bytes) 000496[0] 1761121199.348633 read 0x01381e thru 0x01e565 (0xad48 bytes) 000498[0] 1761121199.389377 trunc from 0x888cf8 to 0x7c809d (0xc0c5a bytes) 000499[1] 1761121199.397030 trunc from 0x7c809e to 0x3278e7 (0x4a07b6 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1199 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x3715 000004[1] 1761121199.644522 trunc from 00000000 to 0x003714 (0x3715 bytes) 000005[1] 1761121199.651687 mapread 0x001000 thru 0x0024b8 (0x14b9 bytes) truncating to largest ever: 0x960575 000006[0] 1761121199.660090 trunc from 0x003715 to 0x960574 (0x95ce60 bytes) 000008[1] 1761121199.669414 mapread 0x439000 thru 0x43a2df (0x12e0 bytes) 000009[1] 1761121199.677860 mapwrite 0x8db000 thru 0x8e6813 (0xb814 bytes) 000011[1] 1761121199.727017 trunc from 0x960575 to 0x5b7f7d (0x3a85f7 bytes) 000012[0] 1761121199.737625 read_OD 0x43f000 thru 0x449fff (0xb000 bytes) 000014[0] 1761121199.746698 read_OD 0x271000 thru 0x278fff (0x8000 bytes) 000017[1] 1761121199.751109 write_OD 0x1e6000 thru 0x1edfff (0x8000 bytes) 000019[0] 1761121199.771857 mapwrite 0x7e9000 thru 0x7eea1c (0x5a1d bytes) 000020[1] 1761121199.852978 mapwrite 0x925000 thru 0x93018f (0xb190 bytes) 000023[0] 1761121199.889879 read_OD 0x3f3000 thru 0x3f9fff (0x7000 bytes) 000026[1] 1761121199.898261 mapwrite 0x696000 thru 0x6a22bd (0xc2be bytes) 000027[0] 1761121199.927578 mapwrite 0x7a3000 thru 0x7aa085 (0x7086 bytes) 000028[0] 1761121199.996852 write_OD 0x758000 thru 0x75afff (0x3000 bytes) 000029[0] 1761121200.015446 mapwrite 0x86a000 thru 0x86ec9e (0x4c9f bytes) 000030[1] 1761121200.039179 read_OD 0x606000 thru 0x60efff (0x9000 bytes) 000031[1] 1761121200.046240 mapread 0x370000 thru 0x37ab40 (0xab41 bytes) 000032[1] 1761121200.053173 trunc from 0x930190 to 0x6f8ee1 (0x2372ae bytes) 000034[0] 1761121200.069879 read_OD 0x658000 thru 0x661fff (0xa000 bytes) 000036[1] 1761121200.079086 write_OD 0x364000 thru 0x36efff (0xb000 bytes) truncating to largest ever: 0x98e664 000038[1] 1761121200.096904 trunc from 0x6f8ee2 to 0x98e663 (0x295782 bytes) 000040[1] 1761121200.117215 mapwrite 0x968000 thru 0x977b61 (0xfb62 bytes) 000042[1] 1761121200.157946 trunc from 0x98e664 to 0x8b6bad (0xd7ab6 bytes) 000043[1] 1761121200.164274 read_OD 0x285000 thru 0x288fff (0x4000 bytes) 000047[0] 1761121200.172025 mapread 0x38d000 thru 0x39bd41 (0xed42 bytes) 000050[1] 1761121200.222365 write_OD 0x2b5000 thru 0x2bdfff (0x9000 bytes) 000052[1] 1761121200.240991 trunc from 0x8b6bae to 0x4df052 (0x3d7b5b bytes) 000053[1] 1761121200.253630 trunc from 0x4df053 to 0x73ac61 (0x25bc0f bytes) 000056[1] 1761121200.270300 mapwrite 0x999000 thru 0x9a2fc3 (0x9fc4 bytes) 000057[0] 1761121200.378072 trunc from 0x9a2fc4 to 0x3baca5 (0x5e831e bytes) 000058[1] 1761121200.391510 mapread 0x222000 thru 0x22a8ea (0x88eb bytes) 000062[1] 1761121200.410873 mapread 0x1dd000 thru 0x1e511e (0x811f bytes) 000063[0] 1761121200.423987 mapread 0x1d2000 thru 0x1d593f (0x3940 bytes) 000064[0] 1761121200.429774 read_OD 0x22c000 thru 0x238fff (0xd000 bytes) 000067[0] 1761121200.433114 mapread 0x0c4000 thru 0x0cb304 (0x7305 bytes) 000068[0] 1761121200.457990 mapwrite 0x02e000 thru 0x033f3a (0x5f3b bytes) 000070[1] 1761121200.496852 mapwrite 0x014000 thru 0x021abc (0xdabd bytes) 000074[1] 1761121200.556713 mapread 0x050000 thru 0x053ca1 (0x3ca2 bytes) 000076[1] 1761121200.562208 mapread 0x2fe000 thru 0x305e50 (0x7e51 bytes) 000079[1] 1761121200.581348 trunc from 0x3baca6 to 0x74bc06 (0x390f61 bytes) 000081[1] 1761121200.592354 mapread 0x369000 thru 0x37821f (0xf220 bytes) 000082[1] 1761121200.619808 mapread 0x257000 thru 0x258a2e (0x1a2f bytes) 000087[1] 1761121200.635109 write_OD 0x709000 thru 0x70cfff (0x4000 bytes) 000088[1] 1761121200.648975 write_OD 0x3f1000 thru 0x3f6fff (0x6000 bytes) 000089[0] 1761121200.663205 trunc from 0x74bc07 to 0x3f3326 (0x3588e0 bytes) 000090[1] 1761121200.670549 mapread 0x1f6000 thru 0x1ff772 (0x9773 bytes) 000093[1] 1761121200.690742 write_OD 0x8a3000 thru 0x8aefff (0xc000 bytes) 000094[0] 1761121200.708074 mapread 0x7e9000 thru 0x7f8941 (0xf942 bytes) 000095[0] 1761121200.750833 mapread 0x450000 thru 0x456d7f (0x6d80 bytes) 000097[1] 1761121200.751275 mapread 0x7cc000 thru 0x7d4587 (0x8588 bytes) 000098[1] 1761121200.764189 mapwrite 0x956000 thru 0x957c3a (0x1c3b bytes) 000099[1] 1761121200.796938 trunc from 0x957c3b to 0x71da57 (0x23a1e3 bytes) 000100[1] 1761121200.806425 mapread 0x5af000 thru 0x5af2bf (0x2c0 bytes) 000101[0] 1761121200.809348 mapwrite 0x34a000 thru 0x34bce4 (0x1ce5 bytes) 000102[1] 1761121200.828653 mapread 0x1a1000 thru 0x1ad379 (0xc37a bytes) 000103[0] 1761121200.848856 trunc from 0x71da58 to 0x0c0ed8 (0x65cb7f bytes) 000104[0] 1761121200.860398 write_OD 0x5f5000 thru 0x5fbfff (0x7000 bytes) 000105[0] 1761121200.875922 mapwrite 0x758000 thru 0x760a40 (0x8a41 bytes) 000106[1] 1761121200.912251 trunc from 0x760a41 to 0x24cf16 (0x513b2a bytes) 000107[0] 1761121200.922126 trunc from 0x24cf17 to 0x8db906 (0x68e9f0 bytes) 000108[0] 1761121200.932166 trunc from 0x8db907 to 0x8138ba (0xc804c bytes) 000111[1] 1761121200.945515 mapread 0x159000 thru 0x15aa13 (0x1a14 bytes) truncating to largest ever: 0x9a130f 000113[0] 1761121200.953234 trunc from 0x8138bb to 0x9a130e (0x18da54 bytes) 000114[0] 1761121200.959707 mapwrite 0x9dc000 thru 0x9e6422 (0xa423 bytes) 000116[1] 1761121200.997808 trunc from 0x9e6423 to 0x4d8488 (0x50df9a bytes) 000117[1] 1761121201.004436 write_OD 0x83c000 thru 0x83dfff (0x2000 bytes) 000119[1] 1761121201.019674 write_OD 0x55d000 thru 0x564fff (0x8000 bytes) 000120[0] 1761121201.032291 trunc from 0x83e000 to 0x757f1a (0xe60e5 bytes) 000121[1] 1761121201.038681 write_OD 0x726000 thru 0x732fff (0xd000 bytes) 000124[0] 1761121201.053567 mapwrite 0x085000 thru 0x08a15e (0x515f bytes) 000125[0] 1761121201.075424 write_OD 0x03d000 thru 0x041fff (0x5000 bytes) 000128[1] 1761121201.089115 trunc from 0x757f1b to 0x34c957 (0x40b5c3 bytes) 000129[0] 1761121201.095722 mapwrite 0x843000 thru 0x846b1a (0x3b1b bytes) 000130[0] 1761121201.125000 write_OD 0x482000 thru 0x48dfff (0xc000 bytes) 000131[1] 1761121201.137532 trunc from 0x846b1b to 0x1863c5 (0x6c0755 bytes) 000133[1] 1761121201.150683 mapwrite 0x04f000 thru 0x04f43c (0x43d bytes) 000134[1] 1761121201.167238 mapwrite 0x792000 thru 0x799705 (0x7706 bytes) 000135[0] 1761121201.202237 read_OD 0x58d000 thru 0x598fff (0xc000 bytes) 000136[0] 1761121201.209052 read_OD 0x46b000 thru 0x46cfff (0x2000 bytes) 000138[0] 1761121201.210603 mapwrite 0x6f1000 thru 0x6f9421 (0x8422 bytes) 000141[0] 1761121201.257906 mapwrite 0x9cf000 thru 0x9d57e9 (0x67ea bytes) 000142[0] 1761121201.289476 mapread 0x983000 thru 0x988faf (0x5fb0 bytes) 000143[0] 1761121201.301418 write_OD 0x1da000 thru 0x1dffff (0x6000 bytes) 000147[0] 1761121201.320485 read_OD 0x784000 thru 0x78ffff (0xc000 bytes) 000149[0] 1761121201.323150 trunc from 0x9d57ea to 0x362be9 (0x672c00 bytes) 000150[0] 1761121201.333952 write_OD 0x4da000 thru 0x4dffff (0x6000 bytes) 000151[0] 1761121201.347684 read_OD 0x064000 thru 0x066fff (0x3000 bytes) 000152[0] 1761121201.349146 mapread 0x49e000 thru 0x4a066f (0x2670 bytes) 000154[0] 1761121201.378147 mapread 0x40a000 thru 0x414cac (0xacad bytes) 000155[0] 1761121201.378506 read_OD 0x3bb000 thru 0x3c4fff (0xa000 bytes) 000156[1] 1761121201.381102 mapwrite 0x61c000 thru 0x625798 (0x9799 bytes) 000158[1] 1761121201.425865 mapwrite 0x8b1000 thru 0x8bafe7 (0x9fe8 bytes) 000160[1] 1761121201.457534 mapwrite 0x736000 thru 0x73a644 (0x4645 bytes) 000162[1] 1761121201.478130 read_OD 0x7df000 thru 0x7ebfff (0xd000 bytes) 000167[0] 1761121201.485895 write_OD 0x4c7000 thru 0x4ccfff (0x6000 bytes) 000171[1] 1761121201.499339 write_OD 0x5fb000 thru 0x608fff (0xe000 bytes) 000172[0] 1761121201.511525 read_OD 0x1f2000 thru 0x1f6fff (0x5000 bytes) 000173[1] 1761121201.514798 write_OD 0x3b4000 thru 0x3bffff (0xc000 bytes) 000174[1] 1761121201.527293 read_OD 0x511000 thru 0x513fff (0x3000 bytes) 000176[1] 1761121201.530490 mapwrite 0x219000 thru 0x228c21 (0xfc22 bytes) 000177[1] 1761121201.595985 read_OD 0x6ce000 thru 0x6d9fff (0xc000 bytes) 000178[0] 1761121201.604093 read_OD 0x1fb000 thru 0x203fff (0x9000 bytes) 000179[0] 1761121201.607217 trunc from 0x8bafe8 to 0x0176b4 (0x8a3933 bytes) 000180[1] 1761121201.615069 mapread 0x004000 thru 0x00f4e3 (0xb4e4 bytes) 000182[0] 1761121201.636874 read_OD 0x009000 thru 0x00bfff (0x3000 bytes) 000183[1] 1761121201.640034 mapwrite 0x8b5000 thru 0x8b7a9d (0x2a9e bytes) 000184[1] 1761121201.678518 trunc from 0x8b7a9e to 0x0af893 (0x80820a bytes) 000186[0] 1761121201.689626 trunc from 0x0af894 to 0x6aea1a (0x5ff187 bytes) 000187[0] 1761121201.701944 trunc from 0x6aea1b to 0x8a3d6f (0x1f5355 bytes) 000189[0] 1761121201.712554 trunc from 0x8a3d70 to 0x7051ee (0x19eb81 bytes) 000193[1] 1761121201.727413 read_OD 0x15d000 thru 0x165fff (0x9000 bytes) 000194[0] 1761121201.735177 mapread 0x127000 thru 0x129ce6 (0x2ce7 bytes) 000197[0] 1761121201.781914 write_OD 0x6b3000 thru 0x6b7fff (0x5000 bytes) 000198[0] 1761121201.803694 mapread 0x67a000 thru 0x682d21 (0x8d22 bytes) 000200[1] 1761121201.816990 mapread 0x63f000 thru 0x64a398 (0xb399 bytes) 000201[0] 1761121201.850941 write_OD 0x615000 thru 0x61efff (0xa000 bytes) 000202[1] 1761121201.869336 read_OD 0x10b000 thru 0x117fff (0xd000 bytes) 000203[1] 1761121201.872881 mapread 0x46c000 thru 0x479d31 (0xdd32 bytes) 000208[0] 1761121201.905263 trunc from 0x7051ef to 0x98faae (0x28a8c0 bytes) 000210[1] 1761121201.923730 mapread 0x440000 thru 0x449202 (0x9203 bytes) 000211[1] 1761121201.956205 read_OD 0x170000 thru 0x17cfff (0xd000 bytes) 000212[0] 1761121201.958815 trunc from 0x98faaf to 0x1de395 (0x7b1719 bytes) 000216[0] 1761121201.982556 mapwrite 0x4e8000 thru 0x4eb136 (0x3137 bytes) 000223[1] 1761121202.024182 mapwrite 0x524000 thru 0x52669b (0x269c bytes) 000224[0] 1761121202.064196 mapwrite 0x49c000 thru 0x4a227d (0x627e bytes) 000232[0] 1761121202.114322 trunc from 0x52669c to 0x40b23e (0x11b45d bytes) 000235[1] 1761121202.127688 read_OD 0x16c000 thru 0x175fff (0xa000 bytes) 000236[0] 1761121202.133810 mapread 0x00b000 thru 0x0111f5 (0x61f6 bytes) 000237[0] 1761121202.143346 mapread 0x3e5000 thru 0x3e8c93 (0x3c94 bytes) 000238[1] 1761121202.148903 mapread 0x239000 thru 0x23d166 (0x4167 bytes) 000239[1] 1761121202.165424 mapwrite 0x3e9000 thru 0x3f6c45 (0xdc46 bytes) 000240[1] 1761121202.220154 mapwrite 0x6b7000 thru 0x6c2669 (0xb66a bytes) 000241[1] 1761121202.259140 mapwrite 0x18e000 thru 0x1914a6 (0x34a7 bytes) 000242[1] 1761121202.277796 mapread 0x223000 thru 0x228bc2 (0x5bc3 bytes) 000244[0] 1761121202.290052 trunc from 0x6c266a to 0x304a53 (0x3bdc16 bytes) 000245[1] 1761121202.300967 read_OD 0x15a000 thru 0x15dfff (0x4000 bytes) 000246[1] 1761121202.307204 read_OD 0x0c5000 thru 0x0d3fff (0xf000 bytes) 000248[1] 1761121202.310615 trunc from 0x304a54 to 0x0a3916 (0x26113d bytes) truncating to largest ever: 0x9a2222 000250[0] 1761121202.317357 trunc from 0x0a3917 to 0x9a2221 (0x8fe90b bytes) 000251[1] 1761121202.329219 mapread 0x6ca000 thru 0x6d56f5 (0xb6f6 bytes) 000252[0] 1761121202.380055 write_OD 0x51f000 thru 0x520fff (0x2000 bytes) 000253[0] 1761121202.400476 trunc from 0x9a2222 to 0x099563 (0x908cbe bytes) 000255[0] 1761121202.412576 mapwrite 0x47a000 thru 0x47e311 (0x4312 bytes) 000256[0] 1761121202.440659 write_OD 0x009000 thru 0x011fff (0x9000 bytes) 000257[0] 1761121202.453519 read_OD 0x417000 thru 0x41bfff (0x5000 bytes) 000258[0] 1761121202.455624 mapwrite 0x6fc000 thru 0x7089e6 (0xc9e7 bytes) 000259[1] 1761121202.501445 read_OD 0x36e000 thru 0x377fff (0xa000 bytes) 000262[0] 1761121202.512741 mapread 0x494000 thru 0x49dd09 (0x9d0a bytes) 000263[0] 1761121202.525149 write_OD 0x59f000 thru 0x5abfff (0xd000 bytes) 000264[1] 1761121202.538303 mapread 0x0ce000 thru 0x0d7dc1 (0x9dc2 bytes) 000265[1] 1761121202.572949 mapread 0x658000 thru 0x6626dc (0xa6dd bytes) 000266[0] 1761121202.573437 mapread 0x37c000 thru 0x383d66 (0x7d67 bytes) 000268[1] 1761121202.584781 trunc from 0x7089e7 to 0x5ea52e (0x11e4b8 bytes) 000272[0] 1761121202.609362 write_OD 0x87c000 thru 0x886fff (0xb000 bytes) 000273[1] 1761121202.622883 mapwrite 0x85d000 thru 0x8623b6 (0x53b7 bytes) 000275[0] 1761121202.645937 write_OD 0x9c7000 thru 0x9d4fff (0xe000 bytes) 000276[0] 1761121202.659224 mapread 0x19d000 thru 0x1ac70d (0xf70e bytes) 000277[1] 1761121202.696496 mapread 0x188000 thru 0x18ceca (0x4ecb bytes) 000279[1] 1761121202.703412 mapwrite 0x651000 thru 0x65f17d (0xe17e bytes) 000284[1] 1761121202.761292 mapwrite 0x8ec000 thru 0x8f8550 (0xc551 bytes) 000285[1] 1761121202.798698 write_OD 0x7e8000 thru 0x7e9fff (0x2000 bytes) 000290[1] 1761121202.812548 mapwrite 0x184000 thru 0x18d08d (0x908e bytes) 000292[1] 1761121202.841031 read_OD 0x922000 thru 0x92ffff (0xe000 bytes) 000293[0] 1761121202.845798 trunc from 0x9d5000 to 0x79bcce (0x239331 bytes) 000294[1] 1761121202.856552 mapwrite 0x5f1000 thru 0x5fe020 (0xd021 bytes) 000295[1] 1761121202.892733 trunc from 0x79bccf to 0x91832a (0x17c65c bytes) 000297[1] 1761121202.902126 read_OD 0x0d3000 thru 0x0defff (0xc000 bytes) 000299[1] 1761121202.906651 read_OD 0x56b000 thru 0x577fff (0xd000 bytes) 000300[0] 1761121202.908786 trunc from 0x91832b to 0x335174 (0x5e31b6 bytes) 000301[1] 1761121202.915202 trunc from 0x335175 to 0x06145a (0x2d3d1a bytes) 000304[0] 1761121202.933011 trunc from 0x06145b to 0x6f9b38 (0x6986de bytes) 000308[1] 1761121202.948824 write_OD 0x7cf000 thru 0x7dcfff (0xe000 bytes) 000309[1] 1761121202.962811 mapread 0x567000 thru 0x56a718 (0x3719 bytes) 000310[0] 1761121202.984441 write_OD 0x275000 thru 0x27efff (0xa000 bytes) 000313[0] 1761121203.001237 read_OD 0x452000 thru 0x455fff (0x4000 bytes) 000314[1] 1761121203.002905 read_OD 0x788000 thru 0x789fff (0x2000 bytes) 000315[1] 1761121203.004424 mapwrite 0x4fb000 thru 0x4fe001 (0x3002 bytes) 000316[1] 1761121203.030681 read_OD 0x6da000 thru 0x6dbfff (0x2000 bytes) 000317[1] 1761121203.034832 mapwrite 0x381000 thru 0x38dad7 (0xcad8 bytes) 000319[0] 1761121203.084276 mapwrite 0x221000 thru 0x223f1c (0x2f1d bytes) 000322[0] 1761121203.137657 read_OD 0x325000 thru 0x328fff (0x4000 bytes) 000324[1] 1761121203.144931 mapread 0x5d0000 thru 0x5d9998 (0x9999 bytes) 000325[0] 1761121203.159363 mapwrite 0x527000 thru 0x536e97 (0xfe98 bytes) 000329[0] 1761121203.226051 read_OD 0x4e1000 thru 0x4e3fff (0x3000 bytes) 000330[1] 1761121203.234326 mapread 0x0a3000 thru 0x0a451a (0x151b bytes) 000331[1] 1761121203.238219 write_OD 0x57e000 thru 0x589fff (0xc000 bytes) 000332[0] 1761121203.256815 trunc from 0x7dd000 to 0x2312da (0x5abd25 bytes) 000333[1] 1761121203.264650 trunc from 0x2312db to 0x635d5d (0x404a83 bytes) truncating to largest ever: 0x9b6d6d 000334[0] 1761121203.276225 trunc from 0x635d5e to 0x9b6d6c (0x38100f bytes) 000335[1] 1761121203.285742 mapread 0x05e000 thru 0x0652ad (0x72ae bytes) 000337[0] 1761121203.314282 write_OD 0x7ea000 thru 0x7f8fff (0xf000 bytes) 000340[1] 1761121203.333351 write_OD 0x338000 thru 0x342fff (0xb000 bytes) 000342[1] 1761121203.345742 trunc from 0x9b6d6d to 0x62ce4b (0x389f21 bytes) 000345[1] 1761121203.357504 read_OD 0x457000 thru 0x45dfff (0x7000 bytes) 000346[1] 1761121203.361871 mapwrite 0x177000 thru 0x17872f (0x1730 bytes) 000351[1] 1761121203.403920 write_OD 0x59e000 thru 0x5a0fff (0x3000 bytes) 000352[0] 1761121203.416827 read_OD 0x2ac000 thru 0x2b6fff (0xb000 bytes) 000354[0] 1761121203.420190 mapwrite 0x643000 thru 0x6481da (0x51db bytes) 000356[0] 1761121203.471061 trunc from 0x6481db to 0x088b3e (0x5bf69c bytes) 000360[1] 1761121203.485344 read_OD 0x04e000 thru 0x053fff (0x6000 bytes) 000361[0] 1761121203.492022 write_OD 0x876000 thru 0x87ffff (0xa000 bytes) 000362[0] 1761121203.506379 read_OD 0x474000 thru 0x475fff (0x2000 bytes) 000363[1] 1761121203.509200 trunc from 0x880000 to 0x16c311 (0x713cee bytes) 000365[0] 1761121203.523164 mapwrite 0x6f0000 thru 0x6f0470 (0x471 bytes) 000367[1] 1761121203.557628 write_OD 0x085000 thru 0x087fff (0x3000 bytes) 000368[1] 1761121203.572868 trunc from 0x6f0471 to 0x0a6688 (0x649de8 bytes) 000369[1] 1761121203.583407 mapread 0x00c000 thru 0x0162dc (0xa2dd bytes) 000371[0] 1761121203.604540 read_OD 0x04b000 thru 0x051fff (0x7000 bytes) 000375[1] 1761121203.612202 read_OD 0x08e000 thru 0x08ffff (0x2000 bytes) 000376[1] 1761121203.613514 write_OD 0x611000 thru 0x61efff (0xe000 bytes) 000378[1] 1761121203.628048 mapwrite 0x152000 thru 0x15bb68 (0x9b69 bytes) 000383[1] 1761121203.683797 read_OD 0x086000 thru 0x08dfff (0x8000 bytes) 000387[0] 1761121203.694545 mapread 0x36d000 thru 0x36d7f7 (0x7f8 bytes) 000390[1] 1761121203.724889 trunc from 0x61f000 to 0x261f05 (0x3bd0fa bytes) 000394[0] 1761121203.748823 write_OD 0x4a2000 thru 0x4a2fff (0x1000 bytes) 000398[0] 1761121203.762339 write_OD 0x76f000 thru 0x772fff (0x4000 bytes) 000399[0] 1761121203.774493 mapread 0x6ea000 thru 0x6ef466 (0x5467 bytes) 000401[1] 1761121203.784257 trunc from 0x773000 to 0x394a84 (0x3de57b bytes) 000402[0] 1761121203.792645 mapwrite 0x110000 thru 0x116827 (0x6828 bytes) 000405[0] 1761121203.824183 write_OD 0x4d1000 thru 0x4d6fff (0x6000 bytes) 000408[1] 1761121203.836667 mapwrite 0x0bd000 thru 0x0ccb29 (0xfb2a bytes) 000410[0] 1761121203.882195 mapread 0x25b000 thru 0x266c67 (0xbc68 bytes) 000412[1] 1761121203.908190 mapread 0x4c0000 thru 0x4c0527 (0x528 bytes) 000415[1] 1761121203.909936 read_OD 0x3cc000 thru 0x3d5fff (0xa000 bytes) 000416[0] 1761121203.912332 mapread 0x149000 thru 0x14cdfe (0x3dff bytes) 000417[0] 1761121203.918256 mapread 0x2e0000 thru 0x2eee06 (0xee07 bytes) 000418[0] 1761121203.937862 mapread 0x1d2000 thru 0x1d4df5 (0x2df6 bytes) 000419[0] 1761121203.941948 read_OD 0x458000 thru 0x45dfff (0x6000 bytes) 000420[1] 1761121203.944263 trunc from 0x4d7000 to 0x210925 (0x2c66da bytes) 000421[1] 1761121203.952036 mapwrite 0x544000 thru 0x551bf3 (0xdbf4 bytes) 000422[1] 1761121203.987396 write_OD 0x6b6000 thru 0x6b8fff (0x3000 bytes) 000424[0] 1761121204.002207 write_OD 0x363000 thru 0x36efff (0xc000 bytes) 000425[0] 1761121204.015287 mapwrite 0x982000 thru 0x9831ad (0x11ae bytes) 000426[0] 1761121204.066366 read_OD 0x7f3000 thru 0x7fcfff (0xa000 bytes) 000430[0] 1761121204.078624 write_OD 0x229000 thru 0x22ffff (0x7000 bytes) 000431[1] 1761121204.092068 read_OD 0x302000 thru 0x30bfff (0xa000 bytes) 000432[1] 1761121204.095620 trunc from 0x9831ae to 0x3e758c (0x59bc21 bytes) 000435[0] 1761121204.112554 mapread 0x293000 thru 0x298200 (0x5201 bytes) truncating to largest ever: 0x9dcc8e 000436[1] 1761121204.124860 trunc from 0x3e758d to 0x9dcc8d (0x5f5701 bytes) 000440[0] 1761121204.139041 mapwrite 0x1b6000 thru 0x1c59b3 (0xf9b4 bytes) 000441[0] 1761121204.195110 trunc from 0x9dcc8e to 0x4722cf (0x56a9be bytes) 000443[1] 1761121204.210348 mapwrite 0x6db000 thru 0x6dd809 (0x280a bytes) 000444[0] 1761121204.249104 trunc from 0x6dd80a to 0x888cde (0x1ab4d5 bytes) 000446[1] 1761121204.265880 mapwrite 0x36a000 thru 0x3735b5 (0x95b6 bytes) 000447[0] 1761121204.304022 trunc from 0x888cdf to 0x60e697 (0x27a647 bytes) 000451[1] 1761121204.321132 mapwrite 0x1a4000 thru 0x1ab25f (0x7260 bytes) 000453[0] 1761121204.357195 mapread 0x07a000 thru 0x07fec6 (0x5ec7 bytes) 000454[0] 1761121204.369521 trunc from 0x60e698 to 0x73c542 (0x12deab bytes) 000456[0] 1761121204.376431 mapwrite 0x501000 thru 0x50a3a8 (0x93a9 bytes) 000459[0] 1761121204.416357 mapread 0x176000 thru 0x17b29d (0x529e bytes) 000462[0] 1761121204.429115 mapwrite 0x45f000 thru 0x46ed72 (0xfd73 bytes) 000464[1] 1761121204.472542 mapwrite 0x1cf000 thru 0x1d246c (0x346d bytes) 000465[0] 1761121204.501699 trunc from 0x73c543 to 0x88674c (0x14a20a bytes) 000466[1] 1761121204.508571 trunc from 0x88674d to 0x0c3483 (0x7c32c9 bytes) 000467[1] 1761121204.516075 mapwrite 0x7a5000 thru 0x7a9d91 (0x4d92 bytes) 000468[0] 1761121204.542101 mapwrite 0x1f6000 thru 0x1fef0c (0x8f0d bytes) 000469[0] 1761121204.576149 trunc from 0x7a9d92 to 0x5ef545 (0x1ba84c bytes) 000471[1] 1761121204.581294 trunc from 0x5ef546 to 0x4deb8c (0x1109b9 bytes) 000473[1] 1761121204.595556 trunc from 0x4deb8d to 0x238372 (0x2a681a bytes) 000474[0] 1761121204.600234 mapwrite 0x322000 thru 0x32eced (0xccee bytes) 000478[1] 1761121204.653658 write_OD 0x216000 thru 0x221fff (0xc000 bytes) 000483[1] 1761121204.668924 write_OD 0x4dd000 thru 0x4e1fff (0x5000 bytes) 000484[1] 1761121204.682080 trunc from 0x4e2000 to 0x52f500 (0x4d501 bytes) 000485[0] 1761121204.688882 mapwrite 0x998000 thru 0x99d36b (0x536c bytes) 000486[0] 1761121204.721214 read_OD 0x51f000 thru 0x521fff (0x3000 bytes) 000490[0] 1761121204.728018 trunc from 0x99d36c to 0x8fc776 (0xa0bf5 bytes) 000491[0] 1761121204.735045 read_OD 0x763000 thru 0x769fff (0x7000 bytes) 000492[1] 1761121204.739385 trunc from 0x8fc777 to 0x0f8c2d (0x803b49 bytes) 000494[1] 1761121204.754952 mapread 0x060000 thru 0x06dffb (0xdffc bytes) 000495[1] 1761121204.775566 write_OD 0x78b000 thru 0x797fff (0xd000 bytes) 000498[1] 1761121204.792947 mapwrite 0x7ce000 thru 0x7cefd4 (0xfd5 bytes) All operations completed A-OK! PASS 16a (14s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 04:20:06 (1761121206) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0888563 s, 23.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0590762 s, 35.5 MB/s Chance of close/open is 1 in 50 Seed set to 1206 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1761121207.136503 write 0x30eb90 thru 0x30f23f (0x6b0 bytes) 000005[0] 1761121207.145444 mapread 0x112278 thru 0x1141b7 (0x1f40 bytes) truncating to largest ever: 0x6b2b92 000006[0] 1761121207.155159 trunc from 0x30f240 to 0x6b2b91 (0x3a3952 bytes) 000008[0] 1761121207.170011 mapread 0x477545 thru 0x477c69 (0x725 bytes) 000009[0] 1761121207.172824 mapread 0x33c30f thru 0x33d3ce (0x10c0 bytes) 000010[1] 1761121207.175721 mapwrite 0x01d0ef thru 0x01e21b (0x112d bytes) 000011[0] 1761121207.211130 mapwrite 0x93a2a9 thru 0x93c280 (0x1fd8 bytes) 000013[0] 1761121207.245477 read 0x50584d thru 0x5068a7 (0x105b bytes) 000014[0] 1761121207.247111 trunc from 0x93c281 to 0x2c1ba7 (0x67a6d9 bytes) 000015[1] 1761121207.256918 trunc from 0x2c1ba8 to 0x1d7ae8 (0xea0bf bytes) 000020[1] 1761121207.289093 mapread 0x14b04c thru 0x14b1a2 (0x157 bytes) 000021[1] 1761121207.291136 write 0x7b4863 thru 0x7b64c1 (0x1c5f bytes) 000023[0] 1761121207.297122 mapread 0x0faef8 thru 0x0fc3c9 (0x14d2 bytes) 000024[0] 1761121207.311332 mapwrite 0x3b4f30 thru 0x3b6460 (0x1531 bytes) 000025[0] 1761121207.341940 mapwrite 0x2653e3 thru 0x265bea (0x808 bytes) truncating to largest ever: 0x97365e 000028[1] 1761121207.368288 trunc from 0x7b64c2 to 0x97365d (0x1bd19c bytes) 000030[0] 1761121207.386611 trunc from 0x97365e to 0x48e8f2 (0x4e4d6b bytes) 000032[0] 1761121207.395529 read 0x2c3d6c thru 0x2c3e8b (0x120 bytes) 000035[1] 1761121207.403182 write 0x2ce099 thru 0x2ceef9 (0xe61 bytes) 000036[0] 1761121207.411262 mapread 0x07070e thru 0x071bf2 (0x14e5 bytes) 000037[1] 1761121207.420530 trunc from 0x48e8f3 to 0x878e4b (0x3ea559 bytes) 000038[1] 1761121207.430824 trunc from 0x878e4c to 0x0e88b7 (0x790594 bytes) 000039[1] 1761121207.435006 write 0x262446 thru 0x2636d8 (0x1293 bytes) 000041[1] 1761121207.441710 mapread 0x02955d thru 0x02a2f3 (0xd97 bytes) 000042[1] 1761121207.449011 write 0x8a3afe thru 0x8a5418 (0x191b bytes) 000043[0] 1761121207.450642 mapwrite 0x709578 thru 0x70b12d (0x1bb6 bytes) 000044[0] 1761121207.489704 read 0x63bf15 thru 0x63c796 (0x882 bytes) 000045[1] 1761121207.514566 mapread 0x677017 thru 0x678097 (0x1081 bytes) 000046[1] 1761121207.520001 read 0x523cf0 thru 0x525218 (0x1529 bytes) 000048[0] 1761121207.526515 write 0x6c3ed7 thru 0x6c5dc6 (0x1ef0 bytes) 000049[0] 1761121207.533200 read 0x591b57 thru 0x591bf5 (0x9f bytes) 000050[0] 1761121207.562341 mapwrite 0x4ed943 thru 0x4ee08d (0x74b bytes) 000051[1] 1761121207.580703 write 0x074140 thru 0x07439c (0x25d bytes) 000052[0] 1761121207.590834 mapread 0x1d5044 thru 0x1d69a7 (0x1964 bytes) 000053[1] 1761121207.596178 write 0x40287b thru 0x402eea (0x670 bytes) 000055[1] 1761121207.598210 read 0x847061 thru 0x84859c (0x153c bytes) 000058[0] 1761121207.612717 mapread 0x19e8f9 thru 0x1a0112 (0x181a bytes) 000059[1] 1761121207.619989 mapwrite 0x3b6547 thru 0x3b6f00 (0x9ba bytes) 000062[0] 1761121207.650777 mapwrite 0x4ecb2b thru 0x4ed121 (0x5f7 bytes) 000064[0] 1761121207.682822 read 0x38447e thru 0x3855f5 (0x1178 bytes) 000068[0] 1761121207.735216 mapread 0x207e15 thru 0x20920f (0x13fb bytes) 000070[0] 1761121207.740400 mapread 0x753410 thru 0x7534dd (0xce bytes) 000074[0] 1761121207.757149 mapread 0x063f3d thru 0x065ddd (0x1ea1 bytes) 000075[0] 1761121207.761471 trunc from 0x8a5419 to 0x4405e0 (0x464e38 bytes) 000077[1] 1761121207.780211 read 0x39e81b thru 0x39f9b8 (0x119e bytes) 000080[0] 1761121207.796475 write 0x23d197 thru 0x23e4c2 (0x132c bytes) 000081[0] 1761121207.799412 read 0x21086e thru 0x2123fb (0x1b8e bytes) 000082[1] 1761121207.813686 write 0x652a9d thru 0x654275 (0x17d9 bytes) 000084[0] 1761121207.819991 trunc from 0x654276 to 0x166fa5 (0x4ed2d0 bytes) 000087[0] 1761121207.841152 read 0x032ba3 thru 0x03386f (0xccd bytes) 000089[0] 1761121207.854600 mapread 0x0a3326 thru 0x0a355f (0x23a bytes) 000090[1] 1761121207.855163 trunc from 0x166fa6 to 0x8cbb85 (0x764be0 bytes) 000094[0] 1761121207.872413 trunc from 0x8cbb86 to 0x6236ba (0x2a84cb bytes) 000095[0] 1761121207.881782 mapwrite 0x2c2e76 thru 0x2c3421 (0x5ac bytes) 000097[1] 1761121207.903532 mapwrite 0x444aec thru 0x44561c (0xb31 bytes) 000100[1] 1761121207.940837 trunc from 0x6236bb to 0x35ca25 (0x2c6c95 bytes) 000101[1] 1761121207.949539 mapwrite 0x391609 thru 0x391a27 (0x41f bytes) 000107[1] 1761121207.986691 write 0x2e346f thru 0x2e4ed8 (0x1a6a bytes) 000108[1] 1761121207.989958 mapwrite 0x4fbb12 thru 0x4fc654 (0xb43 bytes) 000109[1] 1761121208.012137 mapread 0x10583b thru 0x106c9c (0x1462 bytes) 000110[1] 1761121208.021031 read 0x19ace9 thru 0x19c801 (0x1b19 bytes) 000111[1] 1761121208.049982 mapread 0x0082d3 thru 0x00886c (0x59a bytes) 000112[1] 1761121208.056112 read 0x2fec0d thru 0x300a78 (0x1e6c bytes) 000113[1] 1761121208.060530 trunc from 0x4fc655 to 0x77bc2d (0x27f5d9 bytes) 000114[0] 1761121208.066372 trunc from 0x77bc2e to 0x090cea (0x6eaf43 bytes) 000117[1] 1761121208.086745 read 0x01f388 thru 0x01f71d (0x396 bytes) 000118[1] 1761121208.098170 read 0x06ef53 thru 0x06fc18 (0xcc6 bytes) 000120[0] 1761121208.103956 mapwrite 0x8547a5 thru 0x85634c (0x1ba8 bytes) 000121[1] 1761121208.131247 trunc from 0x85634d to 0x1c3176 (0x6931d6 bytes) 000123[0] 1761121208.146197 write 0x76c4d9 thru 0x76ce3c (0x964 bytes) 000125[1] 1761121208.157553 write 0x45675c thru 0x4567f6 (0x9b bytes) 000126[0] 1761121208.162084 write 0x2729ba thru 0x273e09 (0x1450 bytes) 000128[1] 1761121208.176291 write 0x4d2d95 thru 0x4d464a (0x18b6 bytes) 000129[0] 1761121208.184044 trunc from 0x76ce3d to 0x0800a1 (0x6ecd9b bytes) 000130[1] 1761121208.195936 write 0x40962c thru 0x40aaf0 (0x14c5 bytes) 000132[0] 1761121208.207627 mapread 0x234f03 thru 0x236025 (0x1123 bytes) 000133[1] 1761121208.220900 mapread 0x00fd5e thru 0x011cd0 (0x1f73 bytes) 000135[0] 1761121208.231796 trunc from 0x40aaf1 to 0x7b7514 (0x3aca24 bytes) 000136[0] 1761121208.239339 trunc from 0x7b7515 to 0x48eb94 (0x328980 bytes) 000137[0] 1761121208.243055 mapread 0x47703a thru 0x477848 (0x80f bytes) 000138[1] 1761121208.244819 write 0x11cc9d thru 0x11d5f9 (0x95d bytes) 000142[1] 1761121208.266293 read 0x1b8305 thru 0x1b945e (0x115a bytes) 000143[0] 1761121208.275674 write 0x7c73d6 thru 0x7c8b0f (0x173a bytes) 000146[0] 1761121208.285146 mapwrite 0x82f624 thru 0x82fb9b (0x578 bytes) 000147[0] 1761121208.302321 write 0x5a3c69 thru 0x5a443d (0x7d5 bytes) 000148[1] 1761121208.305319 write 0x4ee381 thru 0x4eec22 (0x8a2 bytes) 000150[0] 1761121208.323782 mapread 0x504725 thru 0x50533d (0xc19 bytes) 000151[0] 1761121208.332873 mapread 0x7ecc13 thru 0x7ed473 (0x861 bytes) 000154[1] 1761121208.350273 read 0x02d2d8 thru 0x02d6c4 (0x3ed bytes) 000156[0] 1761121208.366874 mapwrite 0x292229 thru 0x2938c5 (0x169d bytes) 000158[1] 1761121208.398031 mapwrite 0x2103f3 thru 0x211990 (0x159e bytes) 000160[0] 1761121208.424781 mapread 0x7bf725 thru 0x7c04f6 (0xdd2 bytes) 000165[1] 1761121208.451951 trunc from 0x82fb9c to 0x7c61af (0x699ec bytes) 000166[0] 1761121208.461577 write 0x56b8c6 thru 0x56d39f (0x1ada bytes) 000167[0] 1761121208.471127 write 0x801572 thru 0x802429 (0xeb8 bytes) 000168[0] 1761121208.478332 mapwrite 0x811603 thru 0x8135ca (0x1fc8 bytes) 000170[0] 1761121208.501475 write 0x4248a3 thru 0x425946 (0x10a4 bytes) 000171[1] 1761121208.509731 read 0x174903 thru 0x1756d9 (0xdd7 bytes) 000173[1] 1761121208.520058 mapread 0x471230 thru 0x472d0e (0x1adf bytes) 000174[1] 1761121208.561836 read 0x7dc34c thru 0x7dd85d (0x1512 bytes) 000175[0] 1761121208.562731 mapwrite 0x67ff51 thru 0x680abf (0xb6f bytes) 000176[1] 1761121208.584833 read 0x73a528 thru 0x73a6b2 (0x18b bytes) 000177[1] 1761121208.589292 write 0x9e143a thru 0x9e3328 (0x1eef bytes) 000182[1] 1761121208.614896 mapwrite 0x83f843 thru 0x8406d8 (0xe96 bytes) 000184[1] 1761121208.648225 write 0x4bc911 thru 0x4bd12c (0x81c bytes) 000187[1] 1761121208.664119 write 0x8fd2e3 thru 0x8ff22b (0x1f49 bytes) 000190[0] 1761121208.667554 read 0x7b55a8 thru 0x7b5b86 (0x5df bytes) 000192[1] 1761121208.684208 mapwrite 0x707bdd thru 0x70857e (0x9a2 bytes) 000194[0] 1761121208.708473 write 0x82b5d2 thru 0x82c64d (0x107c bytes) 000199[0] 1761121208.744085 write 0x3a6c14 thru 0x3a7273 (0x660 bytes) 000200[1] 1761121208.756691 write 0x995f1d thru 0x997958 (0x1a3c bytes) 000201[0] 1761121208.769245 trunc from 0x9e3329 to 0x2c639e (0x71cf8a bytes) 000205[0] 1761121208.791927 trunc from 0x2c639f to 0x48454b (0x1be1ad bytes) 000206[1] 1761121208.798175 write 0x9cd20a thru 0x9ce66f (0x1466 bytes) 000208[0] 1761121208.813294 read 0x8800dd thru 0x880985 (0x8a9 bytes) 000209[1] 1761121208.822578 trunc from 0x9ce670 to 0x737d8c (0x2968e3 bytes) 000211[1] 1761121208.835422 trunc from 0x737d8d to 0x5d64bc (0x1618d0 bytes) 000212[0] 1761121208.844391 trunc from 0x5d64bd to 0x26275d (0x373d5f bytes) 000215[1] 1761121208.858251 mapwrite 0x59c2b2 thru 0x59c4a6 (0x1f5 bytes) 000216[1] 1761121208.889253 mapread 0x53faa5 thru 0x54019e (0x6fa bytes) 000217[0] 1761121208.896949 write 0x25327f thru 0x254235 (0xfb7 bytes) 000219[1] 1761121208.908361 mapwrite 0x940755 thru 0x94217d (0x1a29 bytes) 000220[1] 1761121208.934448 mapread 0x8769d0 thru 0x87721a (0x84b bytes) 000221[0] 1761121208.943116 mapread 0x8341c7 thru 0x8355d7 (0x1411 bytes) 000222[0] 1761121208.955470 mapwrite 0x1babea thru 0x1bb1ed (0x604 bytes) 000227[1] 1761121208.989649 trunc from 0x94217e to 0x7c8247 (0x179f36 bytes) 000229[0] 1761121209.005425 write 0x7a0448 thru 0x7a170f (0x12c8 bytes) 000230[1] 1761121209.014641 write 0x16189f thru 0x16239c (0xafe bytes) 000231[1] 1761121209.025306 read 0x3ada6b thru 0x3af8c5 (0x1e5b bytes) 000237[0] 1761121209.085729 trunc from 0x7c8248 to 0x21247b (0x5b5dcc bytes) 000239[1] 1761121209.100524 write 0x8ea86b thru 0x8eb3b3 (0xb49 bytes) 000241[0] 1761121209.113996 mapread 0x25ed01 thru 0x25f547 (0x847 bytes) 000243[0] 1761121209.131045 read 0x75595d thru 0x75707a (0x171e bytes) 000244[0] 1761121209.147180 read 0x4920e9 thru 0x493dd3 (0x1ceb bytes) 000246[1] 1761121209.167320 mapwrite 0x9b8bdf thru 0x9ba091 (0x14b3 bytes) 000247[0] 1761121209.226797 write 0x7fe7dd thru 0x7ff812 (0x1036 bytes) 000248[1] 1761121209.241461 trunc from 0x9ba092 to 0x60e3c6 (0x3abccb bytes) 000252[1] 1761121209.266695 trunc from 0x60e3c7 to 0x0c318f (0x54b237 bytes) 000254[1] 1761121209.288051 mapwrite 0x20c625 thru 0x20d378 (0xd54 bytes) 000256[1] 1761121209.327665 mapread 0x130a02 thru 0x130e90 (0x48f bytes) 000258[1] 1761121209.340247 read 0x1dc59e thru 0x1dda18 (0x147b bytes) 000259[1] 1761121209.349177 read 0x08dcb5 thru 0x08ddcc (0x118 bytes) 000261[1] 1761121209.360341 mapwrite 0x2088ab thru 0x209810 (0xf66 bytes) 000262[1] 1761121209.378523 write 0x104f16 thru 0x1056bf (0x7aa bytes) 000263[1] 1761121209.383305 write 0x62e5bf thru 0x62e7a7 (0x1e9 bytes) 000264[0] 1761121209.384724 mapwrite 0x0f3df8 thru 0x0f5dbb (0x1fc4 bytes) 000270[1] 1761121209.429798 mapwrite 0x0e3fec thru 0x0e5c05 (0x1c1a bytes) 000271[0] 1761121209.450056 write 0x45579c thru 0x456f27 (0x178c bytes) 000273[0] 1761121209.462398 write 0x8c0e55 thru 0x8c1573 (0x71f bytes) 000275[0] 1761121209.471879 mapread 0x4a0b52 thru 0x4a14a1 (0x950 bytes) 000276[1] 1761121209.480293 write 0x158758 thru 0x1593bb (0xc64 bytes) 000277[0] 1761121209.484383 write 0x42157a thru 0x422ec2 (0x1949 bytes) 000278[0] 1761121209.490626 mapwrite 0x6a0b0d thru 0x6a2680 (0x1b74 bytes) 000279[0] 1761121209.512673 mapread 0x74f43a thru 0x751359 (0x1f20 bytes) 000281[0] 1761121209.527704 mapread 0x0fd38a thru 0x0fd6dd (0x354 bytes) 000283[1] 1761121209.529518 mapread 0x18f7e0 thru 0x191770 (0x1f91 bytes) 000287[1] 1761121209.548161 write 0x3cda78 thru 0x3cf244 (0x17cd bytes) 000289[0] 1761121209.559132 mapwrite 0x5a420f thru 0x5a59a6 (0x1798 bytes) 000292[0] 1761121209.587959 trunc from 0x8c1574 to 0x035424 (0x88c14f bytes) 000293[0] 1761121209.597879 trunc from 0x035425 to 0x6b52e0 (0x67febc bytes) 000294[1] 1761121209.604216 mapwrite 0x6c93d0 thru 0x6c9a67 (0x698 bytes) 000295[0] 1761121209.626221 mapwrite 0x0ad858 thru 0x0adeb7 (0x660 bytes) 000300[1] 1761121209.665271 mapwrite 0x16aec9 thru 0x16b186 (0x2be bytes) 000303[1] 1761121209.695255 write 0x3cba92 thru 0x3cda1f (0x1f8e bytes) 000305[0] 1761121209.706139 read 0x0ab56f thru 0x0ad465 (0x1ef7 bytes) 000306[1] 1761121209.713942 mapread 0x51b8cc thru 0x51c40e (0xb43 bytes) 000311[0] 1761121209.737958 trunc from 0x6c9a68 to 0x33c6c1 (0x38d3a6 bytes) 000312[1] 1761121209.747023 write 0x5fe8b1 thru 0x5ffef1 (0x1641 bytes) 000313[1] 1761121209.755004 trunc from 0x5ffef2 to 0x3ac6f4 (0x2537fd bytes) 000314[1] 1761121209.765562 mapread 0x33df03 thru 0x33f2be (0x13bc bytes) 000315[1] 1761121209.770771 write 0x5e2a9d thru 0x5e4667 (0x1bcb bytes) 000317[0] 1761121209.780266 write 0x4fff9d thru 0x5002c3 (0x327 bytes) 000318[1] 1761121209.832849 mapwrite 0x29bb79 thru 0x29c66d (0xaf5 bytes) 000319[1] 1761121209.887502 read 0x2eed01 thru 0x2f04da (0x17da bytes) 000322[1] 1761121209.907000 trunc from 0x5e4668 to 0x8eccd9 (0x308672 bytes) 000324[0] 1761121209.927190 trunc from 0x8eccda to 0x3641d1 (0x588b08 bytes) 000325[0] 1761121209.941138 mapwrite 0x81f16b thru 0x8208d9 (0x176f bytes) 000326[1] 1761121209.978723 trunc from 0x8208da to 0x7f6ae0 (0x29df9 bytes) 000327[0] 1761121209.996145 trunc from 0x7f6ae1 to 0x8cdfb7 (0xd74d7 bytes) 000329[0] 1761121210.010340 read 0x1fae97 thru 0x1faffb (0x165 bytes) 000330[0] 1761121210.059943 mapread 0x55badb thru 0x55c85f (0xd85 bytes) 000332[1] 1761121210.064807 write 0x90d369 thru 0x90ec5d (0x18f5 bytes) 000334[1] 1761121210.078179 write 0x93a25a thru 0x93b153 (0xefa bytes) 000335[0] 1761121210.082358 mapread 0x382179 thru 0x383768 (0x15f0 bytes) 000338[1] 1761121210.099190 mapwrite 0x066bfe thru 0x067239 (0x63c bytes) 000340[0] 1761121210.161242 mapwrite 0x557228 thru 0x558851 (0x162a bytes) 000341[0] 1761121210.187582 write 0x455b9b thru 0x456044 (0x4aa bytes) 000342[1] 1761121210.198971 mapwrite 0x183764 thru 0x184516 (0xdb3 bytes) 000343[0] 1761121210.222920 write 0x2c7211 thru 0x2c7565 (0x355 bytes) 000344[1] 1761121210.229592 mapread 0x6db42b thru 0x6db5a8 (0x17e bytes) 000346[1] 1761121210.253405 write 0x521eb2 thru 0x523921 (0x1a70 bytes) 000347[0] 1761121210.258038 mapread 0x06c631 thru 0x06c930 (0x300 bytes) 000348[0] 1761121210.269323 read 0x4fd10c thru 0x4fef29 (0x1e1e bytes) 000349[0] 1761121210.277424 write 0x286fef thru 0x2886f4 (0x1706 bytes) 000350[1] 1761121210.287721 mapwrite 0x94fa96 thru 0x94fd4c (0x2b7 bytes) 000351[1] 1761121210.314238 mapwrite 0x286e43 thru 0x28727c (0x43a bytes) 000353[1] 1761121210.329496 trunc from 0x94fd4d to 0x0356c5 (0x91a687 bytes) 000354[0] 1761121210.338226 mapwrite 0x8af3f6 thru 0x8b12f9 (0x1f04 bytes) 000357[0] 1761121210.397187 mapwrite 0x71aa64 thru 0x71b7ec (0xd89 bytes) 000360[1] 1761121210.418931 mapread 0x6f530a thru 0x6f5ddb (0xad2 bytes) 000364[0] 1761121210.432693 trunc from 0x8b12fa to 0x8dfaf6 (0x2e7fd bytes) 000366[0] 1761121210.450413 trunc from 0x8dfaf7 to 0x323c4d (0x5bbea9 bytes) 000367[1] 1761121210.462539 mapread 0x0e1521 thru 0x0e2468 (0xf48 bytes) 000370[1] 1761121210.478702 trunc from 0x323c4e to 0x10ea34 (0x215219 bytes) truncating to largest ever: 0x9fe36f 000377[0] 1761121210.506095 trunc from 0x10ea35 to 0x9fe36e (0x8ef93a bytes) 000379[0] 1761121210.520105 read 0x35c29f thru 0x35d51c (0x127e bytes) 000381[1] 1761121210.591940 trunc from 0x9fe36f to 0x7acc77 (0x2516f7 bytes) 000383[1] 1761121210.624220 mapread 0x00ba71 thru 0x00cf7e (0x150e bytes) 000384[0] 1761121210.633667 trunc from 0x7acc78 to 0x9484a5 (0x19b82e bytes) 000385[1] 1761121210.649296 mapread 0x80f049 thru 0x8100d8 (0x1090 bytes) 000386[0] 1761121210.661336 mapread 0x1eff7f thru 0x1f0d87 (0xe09 bytes) 000390[0] 1761121210.671119 read 0x2c3aa0 thru 0x2c3d3b (0x29c bytes) 000391[0] 1761121210.717986 mapwrite 0x748d38 thru 0x74aa7f (0x1d48 bytes) 000392[1] 1761121210.735711 write 0x7933b2 thru 0x7944f4 (0x1143 bytes) 000394[1] 1761121210.751558 read 0x4ee12e thru 0x4eec21 (0xaf4 bytes) 000395[1] 1761121210.778892 mapwrite 0x2cb319 thru 0x2cc1b6 (0xe9e bytes) 000396[0] 1761121210.802862 mapwrite 0x61cd71 thru 0x61d634 (0x8c4 bytes) 000397[1] 1761121210.826012 mapwrite 0x194fae thru 0x19517d (0x1d0 bytes) 000398[0] 1761121210.846698 read 0x647cb0 thru 0x64996d (0x1cbe bytes) 000401[0] 1761121210.864541 mapwrite 0x80cdc8 thru 0x80e91d (0x1b56 bytes) 000403[0] 1761121210.891051 mapread 0x2b7372 thru 0x2b92c9 (0x1f58 bytes) 000406[0] 1761121210.909791 trunc from 0x9484a6 to 0x678407 (0x2d009e bytes) 000407[1] 1761121210.922107 read 0x5b92c0 thru 0x5bb058 (0x1d99 bytes) 000409[1] 1761121210.937041 write 0x7c30f2 thru 0x7c312f (0x3e bytes) 000411[1] 1761121210.951092 trunc from 0x7c3130 to 0x3ac7b7 (0x416978 bytes) truncating to largest ever: 0x9fec36 000415[1] 1761121210.972531 trunc from 0x3ac7b8 to 0x9fec35 (0x65247e bytes) 000416[0] 1761121210.980733 write 0x326b8b thru 0x32855e (0x19d4 bytes) 000417[1] 1761121210.999691 mapread 0x0cbad6 thru 0x0cd242 (0x176d bytes) 000418[0] 1761121211.018916 trunc from 0x9fec36 to 0x945c99 (0xb8f9c bytes) 000419[0] 1761121211.031432 read 0x0fdb57 thru 0x0fe94a (0xdf4 bytes) 000420[0] 1761121211.073251 write 0x3723c3 thru 0x37255e (0x19c bytes) 000421[1] 1761121211.078213 mapread 0x148fa5 thru 0x1493f4 (0x450 bytes) 000423[1] 1761121211.101966 trunc from 0x945c9a to 0x2f29fa (0x65329f bytes) 000425[0] 1761121211.124674 write 0x780243 thru 0x781e0f (0x1bcd bytes) 000426[1] 1761121211.133617 mapwrite 0x434fda thru 0x435f21 (0xf48 bytes) 000430[0] 1761121211.172547 mapread 0x228268 thru 0x2297b3 (0x154c bytes) 000431[0] 1761121211.183702 trunc from 0x781e10 to 0x08e286 (0x6f3b89 bytes) 000432[0] 1761121211.192448 trunc from 0x08e287 to 0x45aea3 (0x3ccc1d bytes) 000437[1] 1761121211.209142 read 0x347428 thru 0x34916c (0x1d45 bytes) 000441[0] 1761121211.225823 read 0x333acc thru 0x335891 (0x1dc6 bytes) 000443[1] 1761121211.236134 write 0x7fabb2 thru 0x7fbe7d (0x12cc bytes) 000445[0] 1761121211.250378 mapread 0x1b2879 thru 0x1b39e7 (0x116f bytes) 000447[1] 1761121211.287134 write 0x4a6faf thru 0x4a8733 (0x1785 bytes) 000449[1] 1761121211.322393 mapread 0x4c1067 thru 0x4c2a88 (0x1a22 bytes) 000452[1] 1761121211.327128 trunc from 0x7fbe7e to 0x06bbd2 (0x7902ab bytes) 000455[1] 1761121211.350206 mapread 0x06b690 thru 0x06b6af (0x20 bytes) 000456[0] 1761121211.352121 mapwrite 0x23ebed thru 0x23f57c (0x990 bytes) 000458[0] 1761121211.381607 mapread 0x029351 thru 0x02a42b (0x10db bytes) 000459[0] 1761121211.384544 read 0x04833c thru 0x049248 (0xf0d bytes) 000460[0] 1761121211.396401 mapwrite 0x669c8d thru 0x66a455 (0x7c9 bytes) 000463[1] 1761121211.446559 write 0x186e59 thru 0x1873b7 (0x55f bytes) 000464[1] 1761121211.458683 write 0x283fb8 thru 0x285279 (0x12c2 bytes) 000467[1] 1761121211.475234 write 0x4fca1b thru 0x4fd5d6 (0xbbc bytes) 000471[1] 1761121211.478652 trunc from 0x66a456 to 0x341576 (0x328edf bytes) 000472[1] 1761121211.487962 mapwrite 0x94ae14 thru 0x94b818 (0xa05 bytes) 000474[0] 1761121211.513443 mapwrite 0x3d84d9 thru 0x3d8c8d (0x7b5 bytes) 000476[0] 1761121211.538097 mapwrite 0x93d719 thru 0x93d8e6 (0x1ce bytes) 000478[1] 1761121211.562470 mapwrite 0x47a62c thru 0x47acc9 (0x69e bytes) 000479[1] 1761121211.583281 read 0x842d4a thru 0x844bba (0x1e71 bytes) 000482[1] 1761121211.601940 trunc from 0x94b819 to 0x9c6fc1 (0x7b7a9 bytes) 000484[0] 1761121211.617813 trunc from 0x9c6fc2 to 0x829c89 (0x19d338 bytes) 000486[1] 1761121211.632132 mapwrite 0x5d287a thru 0x5d2bbe (0x345 bytes) 000489[0] 1761121211.665505 read 0x0cd9bd thru 0x0ced01 (0x1345 bytes) 000491[0] 1761121211.677950 mapwrite 0x225c47 thru 0x227bf8 (0x1fb2 bytes) 000492[1] 1761121211.701939 read 0x2e20fe thru 0x2e38fd (0x1800 bytes) 000494[0] 1761121211.736977 mapwrite 0x8fd391 thru 0x8fd517 (0x187 bytes) 000495[0] 1761121211.815367 mapwrite 0x322eda thru 0x3242d7 (0x13fe bytes) 000496[0] 1761121211.833927 trunc from 0x8fd518 to 0x2f5697 (0x607e80 bytes) 000497[0] 1761121211.850776 read 0x14baa4 thru 0x14d727 (0x1c84 bytes) 000498[1] 1761121211.855027 mapwrite 0x887d49 thru 0x8886e3 (0x99b bytes) All operations completed A-OK! PASS 16b (7s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 04:20:13 (1761121213) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 04:20:15 (1761121215) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.166435 s, 12.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.105572 s, 19.9 MB/s ldlm.namespaces.MGC192.168.201.143@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffffa0fe830c7800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffffa0fe8801e800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffffa0fe8801e800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffffa0fe8801e800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.6074 s, 22.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.10243 s, 95.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.892449 s, 117 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.20633 s, 47.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.92488 s, 35.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.34198 s, 78.1 MB/s PASS 16d (18s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 04:20:34 (1761121234) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.117046 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537263 s, 19.5 MB/s PASS 16e (4s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 04:20:37 (1761121237) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 04:21:01 (1761121261) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 04:21:23 (1761121283) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (3s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 04:21:26 (1761121286) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x2325403 truncating to largest ever: 0x2325403 truncating to largest ever: 0x282e29b truncating to largest ever: 0x2325403 truncating to largest ever: 0x282e29b truncating to largest ever: 0x2325403 truncating to largest ever: 0x2d8bfaa truncating to largest ever: 0x282e29b truncating to largest ever: 0x282e29b truncating to largest ever: 0x2d8bfaa truncating to largest ever: 0x2d8bfaa truncating to largest ever: 0x2325403 truncating to largest ever: 0x2d8bfaa truncating to largest ever: 0x282e29b truncating to largest ever: 0x2d8bfaa truncating to largest ever: 0x3bdf026 truncating to largest ever: 0x3bdf026 truncating to largest ever: 0x3bdf026 truncating to largest ever: 0x3bdf026 truncating to largest ever: 0x3bdf026 truncating to largest ever: 0x3eb1faf truncating to largest ever: 0x3eb1faf truncating to largest ever: 0x3eb1faf truncating to largest ever: 0x3eb1faf truncating to largest ever: 0x3eb1faf truncating to largest ever: 0x3f056a0 truncating to largest ever: 0x3f056a0 truncating to largest ever: 0x3f056a0 truncating to largest ever: 0x3f056a0 truncating to largest ever: 0x3f056a0 truncating to largest ever: 0x3fdc431 truncating to largest ever: 0x3fdc431 truncating to largest ever: 0x3fdc431 truncating to largest ever: 0x3fdc431 truncating to largest ever: 0x3fdc431 All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Wed Oct 22 04:21:46 EDT 2025 + fsx 2 done: rc=0 Wed Oct 22 04:21:46 EDT 2025 + fsx 3 done: rc=0 Wed Oct 22 04:21:46 EDT 2025 + fsx 4 done: rc=0 Wed Oct 22 04:21:46 EDT 2025 + fsx 5 done: rc=0 Wed Oct 22 04:21:46 EDT 2025 PASS 16k (22s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 04:21:48 (1761121308) fail_loc=0x8000030a PASS 17 (5s) == sanityn test 18: mmap sanity check =========================================================================================== 04:21:53 (1761121313) mmap test1: basic mmap operation (PASS, 0.018916s) mmap test2: MAP_PRIVATE not write back (PASS, 0.019193s) mmap test3: concurrent mmap ops on two nodes (PASS, 5.9284s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0895s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.031471s) mmap test6: check mmap write/read content on two nodes (PASS, 0.0346s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.016s) PASS 18 (12s) == sanityn test 19: test concurrent uncached read races ========================================================================= 04:22:05 (1761121325) oleg143-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg143-client: oleg143-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 04:22:07 (1761121327) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 04:22:09 (1761121329) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 04:22:11 (1761121331) now is 1761121332 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 new atime is 1761121393 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 04:23:15 (1761121395) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 17408 7521280 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 10240 3759104 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 17408 7521280 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.2K 384 489.8K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.1K 373 114.8K 1% /mnt/lustre[OST:1] filesystem_summary: 230.0K 384 229.7K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.2K 384 489.8K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.1K 373 114.8K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.0K 384 229.7K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 10.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 17.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 501970 384 501586 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117909 373 117536 1% /mnt/lustre2[OST:1] filesystem_summary: 235552 384 235168 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 17408 7521280 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.2K 384 489.8K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.1K 373 114.8K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.0K 384 229.7K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 501970 384 501586 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 117909 373 117536 1% /mnt/lustre[OST:1] filesystem_summary: 235552 384 235168 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 501970 384 501586 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117909 373 117536 1% /mnt/lustre2[OST:1] filesystem_summary: 235552 384 235168 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 17408 7521280 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 10240 3759104 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 17408 7521280 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 04:23:17 (1761121397) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 04:23:20 (1761121400) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 04:23:23 (1761121403) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 04:23:24 (1761121404) utime: good mknod times 1761121403 <= 1761121404 <= 1761121404 for /mnt/lustre/f26a utime: good mknod times 1761121403 <= 1761121404 <= 1761121404 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 04:23:27 (1761121407) PASS 26b (5s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 04:23:33 (1761121413) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 04:23:37 (1761121417) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 04:23:40 (1761121420) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg143-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.143@tcp:/lustre 7542784 18432 7518208 1% /mnt/lustre2 PASS 31b (14s) == sanityn test 31r: open-rename(replace) race =========== 04:23:54 (1761121434) -rw-r--r-- 1 root root 0 Oct 22 04:23 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 04:24:00 (1761121440) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 04:24:01 (1761121441) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 04:24:02 (1761121442) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 04:24:04 (1761121444) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 04:24:05 (1761121445) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg143-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0fe830c7800.ost_server_uuid,osc.lustre-OST0000-osc-ffffa0fe8801e800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffffa0fe830c7800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffffa0fe8801e800.ost_server_uuid in FULL state after 0 sec oleg143-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0fe830c7800.ost_server_uuid,osc.lustre-OST0001-osc-ffffa0fe8801e800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffffa0fe830c7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffffa0fe8801e800.ost_server_uuid in IDLE state after 0 sec oleg143-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0fe830c7800.ost_server_uuid,osc.lustre-OST0000-osc-ffffa0fe8801e800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffffa0fe830c7800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffffa0fe8801e800.ost_server_uuid in FULL state after 0 sec oleg143-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0fe830c7800.ost_server_uuid,osc.lustre-OST0001-osc-ffffa0fe8801e800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffffa0fe830c7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffffa0fe8801e800.ost_server_uuid in IDLE state after 0 sec oleg143-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0fe830c7800.ost_server_uuid,osc.lustre-OST0000-osc-ffffa0fe8801e800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffffa0fe830c7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffffa0fe8801e800.ost_server_uuid in FULL state after 0 sec oleg143-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0fe830c7800.ost_server_uuid,osc.lustre-OST0001-osc-ffffa0fe8801e800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffffa0fe830c7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffffa0fe8801e800.ost_server_uuid in IDLE state after 0 sec PASS 34 (83s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 04:25:28 (1761121528) fail_loc=0x80000317 Race attempt 0 Wait for 45193 45324 for 60 sec... - open/close 2544 (time 1761121541.39 total 10.00 last 254.40) total: 4000 open/close in 16.28 seconds: 245.64 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.143@tcp:/lustre 7.2G 19M 7.2G 1% /mnt/lustre 192.168.201.143@tcp:/lustre 7.2G 19M 7.2G 1% /mnt/lustre2 PASS 35 (67s) == sanityn test 36: handle ESTALE/open-unlink correctly == 04:26:35 (1761121595) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg143-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.620209 s, 84.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.561925 s, 93.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.550655 s, 95.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.527384 s, 99.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.574369 s, 91.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.53579 s, 97.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.524945 s, 99.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.553496 s, 94.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.461576 s, 114 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.524286 s, 100 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.571293 s, 91.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7519232) after_dd(7468032) after(7519232) PASS 36 (249s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 04:30:44 (1761121844) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 - create 4348 (time 1761121856.04 total 10.00 last 434.74) - create 8684 (time 1761121866.04 total 20.00 last 433.57) total: 10000 create in 22.31 seconds: 448.23 ops/second PASS 37 (27s) == sanityn test 39a: test from 11063 ============================================================================================ 04:31:11 (1761121871) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 04:31:14 (1761121874) repeat after cancel_lru_locks PASS 39b (4s) == sanityn test 39c: check truncate mtime update ================================================================================ 04:31:18 (1761121878) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 04:31:22 (1761121882) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 40a: pdirops: create vs others ======================================================================== 04:31:25 (1761121885) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 04:31:34 (1761121894) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (10s) == sanityn test 40c: pdirops: link and others ======================================================================== 04:31:44 (1761121904) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (10s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 04:31:54 (1761121914) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (10s) == sanityn test 40e: pdirops: rename and others ======================================================================== 04:32:04 (1761121924) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (9s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 04:32:13 (1761121933) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (7s) == sanityn test 41b: pdirops: create vs create ======================================================================== 04:32:20 (1761121940) open: File exists Conflict PASS 41b (7s) == sanityn test 41c: pdirops: create vs link ======================================================================== 04:32:27 (1761121947) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (7s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 04:32:34 (1761121954) Conflict PASS 41d (7s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 04:32:41 (1761121961) Conflict PASS 41e (7s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 04:32:48 (1761121968) Conflict PASS 41f (7s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 04:32:55 (1761121975) Conflict PASS 41g (7s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 04:33:02 (1761121982) Conflict PASS 41h (6s) == sanityn test 41i: reint_open: create vs create ======== 04:33:08 (1761121988) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (999s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 04:49:47 (1761122987) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (12s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 04:49:59 (1761122999) open: File exists Conflict PASS 42b (12s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 04:50:11 (1761123011) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (11s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 04:50:22 (1761123022) Conflict PASS 42d (10s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 04:50:32 (1761123032) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (12s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 04:50:44 (1761123044) Conflict PASS 42f (11s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 04:50:55 (1761123055) Conflict PASS 42g (11s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 04:51:07 (1761123067) Conflict PASS 42h (12s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 04:51:18 (1761123078) PASS 43a (46s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 04:52:04 (1761123124) Conflict PASS 43b (10s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 04:52:14 (1761123134) Conflict PASS 43c (11s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 04:52:25 (1761123145) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (10s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 04:52:35 (1761123155) Conflict PASS 43e (10s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 04:52:45 (1761123165) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (11s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 04:52:56 (1761123176) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (10s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 04:53:06 (1761123186) Conflict PASS 43h (11s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 04:53:17 (1761123197) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (3s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 04:53:20 (1761123200) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (108s) == sanityn test 43k: unlink vs create ==================== 04:55:08 (1761123308) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (974s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 05:11:23 (1761124283) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Oct 22 05:11:31 EDT 2025 PASS 44a (13s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 05:11:36 (1761124296) open: File exists Conflict PASS 44b (12s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 05:11:48 (1761124308) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (14s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 05:12:02 (1761124322) Conflict PASS 44d (14s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 05:12:16 (1761124336) Conflict PASS 44e (13s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 05:12:30 (1761124350) Conflict PASS 44f (14s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 05:12:43 (1761124363) Conflict PASS 44g (14s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 05:12:57 (1761124377) Conflict PASS 44h (12s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 05:13:09 (1761124389) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (3s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 05:13:12 (1761124392) PASS 45a (105s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 05:14:57 (1761124497) Conflict PASS 45b (12s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 05:15:10 (1761124510) Conflict PASS 45c (14s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 05:15:24 (1761124524) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (13s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 05:15:38 (1761124538) Conflict PASS 45e (13s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 05:15:51 (1761124551) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (14s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 05:16:05 (1761124565) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (13s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 05:16:18 (1761124578) Conflict PASS 45h (14s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 05:16:32 (1761124592) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (3s) == sanityn test 45j: read vs rename ====================== 05:16:35 (1761124595) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (1088s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 05:34:44 (1761125684) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (15s) == sanityn test 46b: pdirops: link vs create ======================================================================== 05:35:00 (1761125700) open: File exists Conflict PASS 46b (15s) == sanityn test 46c: pdirops: link vs link ======================================================================== 05:35:15 (1761125715) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (16s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 05:35:31 (1761125731) Conflict PASS 46d (16s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 05:35:48 (1761125748) Conflict PASS 46e (21s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 05:36:08 (1761125768) Conflict PASS 46f (19s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 05:36:27 (1761125787) Conflict PASS 46g (22s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 05:36:49 (1761125809) Conflict PASS 46h (22s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 05:37:11 (1761125831) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (4s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 05:37:15 (1761125835) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (4s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 05:37:20 (1761125840) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (3s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 05:37:24 (1761125844) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (3s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 05:37:28 (1761125848) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (4s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 05:37:31 (1761125851) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (4s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 05:37:35 (1761125855) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (5s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 05:37:40 (1761125860) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (6s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 05:37:46 (1761125866) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.105946 s, 96.7 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (15s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 05:38:01 (1761125881) 0+1 records in 0+1 records out 158 bytes copied, 0.0253366 s, 6.2 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3178: kill: (243208) - No such process PASS 51a (11s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 05:38:12 (1761125892) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0366884 s, 27.9 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154128 s, 6.8 MB/s 3145728 PASS 51b (22s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 05:38:34 (1761125914) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0299501 s, 34.2 kB/s PASS 51c (14s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 05:38:48 (1761125928) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303546 s, 34.5 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (9s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 05:38:57 (1761125937) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2436 0x984 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2436 0x984 0 read lease(1) has applied. PASS 51e (10s) == sanityn test 54: rename locking ======================= 05:39:07 (1761125947) llite.lustre-ffffa0fe830c7800.xattr_cache=1 llite.lustre-ffffa0fe8801e800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504977 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:39:09.000000000 -0400 Modify: 2025-10-22 05:39:09.000000000 -0400 Change: 2025-10-22 05:39:09.000000000 -0400 Birth: 2025-10-22 05:39:08.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504980 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:39:15.000000000 -0400 Modify: 2025-10-22 05:39:16.000000000 -0400 Change: 2025-10-22 05:39:16.000000000 -0400 Birth: 2025-10-22 05:39:15.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504983 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:39:22.000000000 -0400 Modify: 2025-10-22 05:39:22.000000000 -0400 Change: 2025-10-22 05:39:22.000000000 -0400 Birth: 2025-10-22 05:39:21.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504982 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:39:22.000000000 -0400 Modify: 2025-10-22 05:39:23.000000000 -0400 Change: 2025-10-22 05:39:23.000000000 -0400 Birth: 2025-10-22 05:39:21.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504985 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:39:28.000000000 -0400 Modify: 2025-10-22 05:39:29.000000000 -0400 Change: 2025-10-22 05:39:29.000000000 -0400 Birth: 2025-10-22 05:39:28.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504986 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:39:28.000000000 -0400 Modify: 2025-10-22 05:39:29.000000000 -0400 Change: 2025-10-22 05:39:29.000000000 -0400 Birth: 2025-10-22 05:39:28.000000000 -0400 PASS 54 (33s) == sanityn test 55a: rename vs unlink target dir ========= 05:39:40 (1761125980) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (13s) == sanityn test 55b: rename vs unlink source dir ========= 05:39:54 (1761125994) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (14s) == sanityn test 55c: rename vs unlink orphan target dir == 05:40:08 (1761126008) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (19s) == sanityn test 55d: rename file vs link ================= 05:40:28 (1761126028) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (16s) == sanityn test 60: Verify data_version behaviour ======== 05:40:43 (1761126043) PASS 60 (8s) == sanityn test 70a: cd directory && rm directory ======== 05:40:51 (1761126051) PASS 70a (7s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 05:40:58 (1761126058) PASS 70b (8s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 05:41:06 (1761126066) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00709076 s, 9.2 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 05:41:09 (1761126069) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0233771 s, 2.8 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (4s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 05:41:13 (1761126073) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 05:41:16 (1761126076) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0195232 s, 3.4 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (3s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 05:41:19 (1761126079) llite.lustre-ffffa0fe830c7800.xattr_cache=1 llite.lustre-ffffa0fe8801e800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (7s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 05:41:27 (1761126087) llite.lustre-ffffa0fe830c7800.xattr_cache=1 llite.lustre-ffffa0fe8801e800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (7s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 05:41:34 (1761126094) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0116196 s, 88.1 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0522126 s, 19.6 kB/s 253258: get lock2 253263: get lock1 253258: done 253258: get lock1 253263: done 253263: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 253263: cannot get lock2: Resource deadlock avoided 253263: exit rc=0 253258: done 253258: exit rc=0 PASS 74 (11s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 05:41:45 (1761126105) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.136351 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687627 s, 15.2 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3746: 253838 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (13s) == sanityn test 76: Verify MDT open_files listing ======== 05:41:58 (1761126118) opening files: ........ closing files: ........ PASS 76 (73s) == sanityn test 77a: check FIFO NRS policy =============== 05:43:11 (1761126191) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.444303 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0530981 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690129 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102908 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106512 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640589 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205069 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732352 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218462 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129208 s, 8.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358399 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313498 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37264 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.420777 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234198 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315493 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.487182 s, 2.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610061 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108144 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718279 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159628 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111845 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751851 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104292 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175754 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828619 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157049 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857275 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882833 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908445 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586094 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0519372 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435638 s, 24.1 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (10s) == sanityn test 77b: check CRR-N NRS policy ============== 05:43:21 (1761126201) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.351183 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969761 s, 10.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.069043 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056861 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068488 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111874 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170313 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137335 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222756 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175343 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229382 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311949 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369279 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353856 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385277 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.388736 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144238 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11929 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114751 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105928 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102181 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131385 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168483 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848458 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131923 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164644 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116016 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11321 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119147 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139516 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169916 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118957 s, 8.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.504808 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108942 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123384 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347584 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359197 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336366 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.327976 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383208 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311953 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385081 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337713 s, 3.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334989 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387092 s, 2.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409472 s, 2.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.407959 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369164 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.413108 s, 2.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300672 s, 34.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339082 s, 30.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00258728 s, 405 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488467 s, 21.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506575 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109342 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00294335 s, 356 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377559 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0502482 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102802 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661642 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848246 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693211 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957646 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332186 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811093 s, 12.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (13s) == sanityn test 77c: check ORR NRS policy ================ 05:43:34 (1761126214) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.458684 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553061 s, 19.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667213 s, 15.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781667 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889273 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173034 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178666 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159197 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268848 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214186 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286464 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296304 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216523 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237091 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2845 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264954 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319763 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568678 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627186 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610085 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637075 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1035 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706722 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917821 s, 11.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794713 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0628068 s, 16.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119682 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927384 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897778 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128775 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12535 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0750646 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121277 s, 8.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=5 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.511419 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541968 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196472 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869117 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413717 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138269 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952268 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796471 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456442 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126893 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245286 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291984 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188769 s, 5.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323486 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344422 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265909 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264943 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490724 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791449 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0031245 s, 336 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257307 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00380253 s, 276 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30549 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237413 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238551 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151765 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24831 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875612 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738731 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102749 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339162 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0810302 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717116 s, 14.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.360383 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634621 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060553 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159722 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187294 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177373 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203461 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266836 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328907 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321393 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.33442 s, 3.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318626 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.364223 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314878 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3924 s, 2.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381039 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.424813 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891661 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356083 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0474913 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871427 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184957 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172939 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741311 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800389 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829422 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212037 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208947 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216569 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928434 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948041 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215043 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203953 s, 5.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (19s) == sanityn test 77d: check TRR nrs policy ================ 05:43:54 (1761126234) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.639871 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670908 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100087 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120047 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138531 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148669 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127893 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136144 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105907 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526521 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19624 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242507 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257984 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241562 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269784 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299872 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276531 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465052 s, 22.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252348 s, 41.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250016 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580707 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332385 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0451359 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377137 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403963 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043293 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476764 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515831 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05611 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0614822 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11297 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0955508 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0688085 s, 15.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.533292 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068218 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108246 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640603 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177723 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730931 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153119 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238744 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254396 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202744 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209595 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168939 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20401 s, 5.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283986 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205549 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0528082 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867451 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663197 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553596 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562875 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941652 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535484 s, 19.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644843 s, 16.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740757 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730282 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811063 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122056 s, 8.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844039 s, 12.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911169 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822916 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793611 s, 13.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104335 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104276 s, 10.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.460323 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175034 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852577 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992756 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130139 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0672002 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106983 s, 9.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190429 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934298 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245689 s, 4.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354366 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349609 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.434237 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.42129 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.456302 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.420559 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.472881 s, 2.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200912 s, 5.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668001 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108035 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136065 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120242 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150319 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121441 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132658 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150998 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151655 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136698 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173451 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175545 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180524 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178841 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163355 s, 6.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (18s) == sanityn test 77e: check TBF NID nrs policy ============ 05:44:11 (1761126251) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.43@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.30831 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826942 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0672165 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32946 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299152 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.366397 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297839 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306513 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.384065 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37454 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386477 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379875 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363976 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.36362 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372831 s, 2.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.441979 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.410947 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564587 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11104 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822501 s, 127 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626073 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894408 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116074 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617612 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569728 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653706 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143902 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113172 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391891 s, 26.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604166 s, 17.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350493 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312149 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029242 s, 35.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.548231 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623782 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133075 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273669 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272021 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310296 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281327 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308663 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290663 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325751 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.360959 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.384778 s, 2.7 MB/sldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=4 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378585 s, 2.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333731 s, 3.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.39427 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.432129 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.444274 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00365109 s, 287 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0866773 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00285294 s, 368 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00285383 s, 367 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160527 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257497 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291388 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171469 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174985 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152538 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149781 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361697 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185112 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253802 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188024 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481704 s, 21.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.305379 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538255 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155745 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157721 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159582 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185466 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175053 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207143 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189054 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170574 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203601 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16335 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189416 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158398 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12163 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160819 s, 6.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185533 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217735 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255103 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114175 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853577 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122842 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00297528 s, 352 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115868 s, 90.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141982 s, 7.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111256 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17816 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214957 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640136 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012251 s, 85.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435664 s, 24.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355347 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230795 s, 45.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.600549 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811308 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957181 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068886 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139368 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222139 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237762 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244686 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258975 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285669 s, 3.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215474 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318797 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273617 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.33413 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304111 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276876 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321952 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101267 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101514 s, 10.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068334 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888571 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106643 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109024 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11508 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882619 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937889 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172873 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998341 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10364 s, 10.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113875 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577044 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110605 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115453 s, 9.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (25s) == sanityn test 77f: check TBF JobID nrs policy ========== 05:44:36 (1761126276) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.474597 s, 35.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707757 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828307 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059582 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101582 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224221 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15201 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271164 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278857 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201992 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32264 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231972 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268537 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270646 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318588 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291338 s, 3.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395973 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0024904 s, 421 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136803 s, 76.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137606 s, 76.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11353 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0858131 s, 12.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156081 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175782 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142264 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585 s, 17.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701379 s, 15.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953223 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0492576 s, 21.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559391 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497699 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087395 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111359 s, 9.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.295207 s, 56.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933913 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089257 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189194 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186907 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234763 s, 4.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296232 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330918 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295683 s, 3.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26396 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328458 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353624 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302403 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273308 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290635 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303391 s, 3.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385819 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320247 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620132 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148084 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00234234 s, 448 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107314 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906848 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053955 s, 19.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0570515 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0035516 s, 295 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574879 s, 18.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116002 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102876 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647783 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651084 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12572 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142803 s, 7.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.448827 s, 37.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453341 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919176 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11746 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867925 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214239 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188112 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287005 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251928 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30857 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267432 s, 3.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288259 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.36528 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34936 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357864 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299648 s, 3.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.391849 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136294 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0030464 s, 344 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190302 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178728 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149774 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00470566 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135836 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230719 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13339 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142537 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989183 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0528461 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640065 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500613 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435693 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399559 s, 26.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.495709 s, 33.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261477 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120687 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164758 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180107 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127527 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227687 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251568 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260741 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213316 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320958 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191668 s, 5.5 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.351186 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330919 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334119 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233713 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.375074 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361078 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305495 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366255 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104768 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00517608 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00279357 s, 375 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122586 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0033041 s, 317 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0824863 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115059 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556036 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560294 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051298 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0451807 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415359 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0467604 s, 22.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (26s) == sanityn test 77g: Change TBF type directly ============ 05:45:02 (1761126302) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (15s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 05:45:17 (1761126317) oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg143-client: oleg143-server: ssh exited with exit code 2 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg143-client: oleg143-server: ssh exited with exit code 12 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg143-client: oleg143-server: ssh exited with exit code 12 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg143-client: oleg143-server: ssh exited with exit code 12 PASS 77h (14s) == sanityn test 77i: Change rank of TBF rule ============= 05:45:31 (1761126331) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (27s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 05:45:59 (1761126359) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.410035 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583649 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.064122 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055348 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148727 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162908 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245673 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208254 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244731 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208083 s, 5.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233266 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217411 s, 4.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221781 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215629 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23873 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291119 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689004 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736159 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927221 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706778 s, 14.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799284 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108001 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981995 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798982 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678621 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091965 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122633 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129409 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119992 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104465 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114156 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105929 s, 9.9 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.21022 s, 12.8 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6588 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (66s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 05:47:04 (1761126424) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.841225 s, 19.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791426 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529602 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100353 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791264 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134636 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769323 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651822 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0672929 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183521 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27551 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313201 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37612 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399385 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.427282 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.462197 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.449654 s, 2.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983131 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283943 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433523 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462672 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144393 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368581 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395726 s, 26.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525338 s, 20.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536315 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055217 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586138 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673581 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613337 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711709 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278569 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259979 s, 40.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9317 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.973 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.797959 s, 21.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795863 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845969 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161206 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222908 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175159 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219303 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248112 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20324 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250027 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237316 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.362446 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395425 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378492 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.422664 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.401629 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.458505 s, 2.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00282778 s, 371 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00274549 s, 382 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117503 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14053 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155954 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927831 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101405 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779496 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139209 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117983 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759361 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583948 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601745 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0463959 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489276 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409653 s, 25.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8614 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0114 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (153s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 05:49:37 (1761126577) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.534213 s, 31.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973946 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118369 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164956 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174798 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195871 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186881 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293909 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363144 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315163 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382654 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.33372 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.4006 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.396296 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383799 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.408284 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.470417 s, 2.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368902 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102636 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126145 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11876 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110123 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884075 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944033 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194714 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136504 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018929 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687564 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774405 s, 13.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678721 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409571 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561084 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057515 s, 18.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.29106 s, 11.3 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.97944 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.43@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.43@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.402587 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582078 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0474218 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066294 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701186 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561167 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11121 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151432 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138458 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116564 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137033 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235828 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245142 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274144 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247753 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273614 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294945 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615252 s, 17.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363341 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380369 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648088 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751882 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704755 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0519608 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074212 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061058 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683635 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664697 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680085 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634382 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770147 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816912 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925746 s, 11.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.66622 s, 13.7 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.94215 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.43@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.586344 s, 28.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666858 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836756 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842345 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194208 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220524 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14298 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197907 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254225 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188378 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25075 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17115 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279606 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291889 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293207 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284097 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.348281 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184036 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127633 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101931 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162932 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122334 s, 8.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220619 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234187 s, 44.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093889 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880655 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794113 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0646914 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427825 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068189 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11191 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658394 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719093 s, 14.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.42276 s, 12.4 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.07314 s, 20.7 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.532041 s, 31.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992555 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993381 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168899 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201377 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142839 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22392 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278929 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216593 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312762 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309701 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297791 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313284 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318758 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385693 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.527955 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.479049 s, 2.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00507031 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598729 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085803 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657313 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11826 s, 8.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121811 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942805 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0430641 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0461748 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737036 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392832 s, 26.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324992 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020812 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166305 s, 63.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324731 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283812 s, 36.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 12.4146 s, 8.4 MB/s Write runtime is 14 s, speed is 7.142857 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.99621 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.00431 s, 13.1 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9729 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.613086 s, 27.4 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122505 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145657 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277944 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264515 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286231 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329668 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298981 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.377004 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.351487 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.421893 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.408534 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.41318 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.475865 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.436629 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.534118 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.509043 s, 2.1 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982586 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242927 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100822 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00361077 s, 290 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192497 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121106 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00379436 s, 276 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504965 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906427 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141884 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00529913 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657161 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137766 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971509 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852311 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233963 s, 44.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9777 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6633 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.631125 s, 26.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108465 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11865 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108242 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257687 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288974 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290568 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230209 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31321 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353945 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.417543 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.429496 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393864 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.424439 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.479575 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.427894 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.587294 s, 1.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00505596 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105223 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522528 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150715 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019394 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453238 s, 23.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100466 s, 104 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208554 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0038757 s, 271 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00447398 s, 234 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307807 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122027 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456552 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028702 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177849 s, 59.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.06627 s, 11.6 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.8808 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.518175 s, 32.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12593 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624149 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145329 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952919 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233385 s, 4.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280496 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28185 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299166 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329497 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335192 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.370995 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332464 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.442126 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.448593 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.426464 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.495413 s, 2.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011304 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448625 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0457277 s, 22.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199399 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552012 s, 19.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983145 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060691 s, 17.3 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637184 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479776 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128405 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330416 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360101 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722286 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801134 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0542998 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587238 s, 17.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg143-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg143-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.8951 s, 9.6 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.94855 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.6555 s, 9.8 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.92898 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (441s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 05:56:58 (1761127018) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (12s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 05:57:11 (1761127031) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg143-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.52933 s, 297 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg143-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.1478 s, 261 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (59s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 05:58:10 (1761127090) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.571904 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11894 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143082 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155434 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125808 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144541 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150537 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191113 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144985 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114161 s, 9.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0fe830c7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317275 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260161 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278206 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337497 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2611 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297567 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.425453 s, 2.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0fe830c7800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403977 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507168 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508963 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557604 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722015 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759787 s, 13.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776125 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835747 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790279 s, 13.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142362 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920776 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103088 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091268 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130219 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115976 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228831 s, 4.6 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.544 s, 11.0 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.18112 s, 20.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.443612 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677073 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980169 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878236 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102722 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120298 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148365 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162891 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209072 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084732 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17189 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267464 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37978 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349563 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320524 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416184 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.431385 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270558 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00458947 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00318506 s, 329 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0026056 s, 402 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00439273 s, 239 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105757 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145083 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00248205 s, 422 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156722 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158951 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963203 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149149 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144435 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212693 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456638 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210184 s, 49.9 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg143-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg143-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1446 s, 10.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.23306 s, 20.0 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (94s) == sanityn test 77o: Changing rank should not panic ====== 05:59:44 (1761127184) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (13s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 05:59:57 (1761127197) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (130s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 06:02:07 (1761127327) mds.MDS.mdt.nrs_policies=tbf oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg143-client: oleg143-server: ssh exited with exit code 36 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg143-client: oleg143-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (41s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 06:02:48 (1761127368) oleg143-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg143-client: oleg143-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (12s) == sanityn test 79: xattr: intent error ================== 06:03:00 (1761127380) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505445 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 06:03:02.000000000 -0400 Modify: 2025-10-22 06:03:02.000000000 -0400 Change: 2025-10-22 06:03:02.000000000 -0400 Birth: 2025-10-22 06:03:02.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (19s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 06:03:20 (1761127400) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (3s) == sanityn test 80b: Accessing directory during migration ========================================================== 06:03:24 (1761127404) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (3s) == sanityn test 81a: rename and stat under striped directory ========================================================== 06:03:27 (1761127407) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (4s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 06:03:31 (1761127411) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (4s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 06:03:35 (1761127415) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (3s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 06:03:38 (1761127418) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7118 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7118 PASS 82 (8s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 06:03:46 (1761127426) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (3s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 06:03:50 (1761127430) fail_loc=0x8000060b PASS 84 (14s) == sanityn test 90: open/create and unlink striped directory ========================================================== 06:04:03 (1761127443) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (3s) == sanityn test 91: chmod and unlink striped directory === 06:04:06 (1761127446) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (4s) == sanityn test 92: create remote directory under orphan directory ========================================================== 06:04:10 (1761127450) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (3s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 06:04:13 (1761127453) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2651 0xa5b 0 1 2632 0xa48 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2652 0xa5c 0 1 2633 0xa49 0 rc1=0 and rc2=0 PASS 93 (16s) == sanityn test 94: signal vs CP callback race =========== 06:04:29 (1761127469) osc.lustre-OST0000-osc-ffffa0fe830c7800.idle_timeout=0 osc.lustre-OST0000-osc-ffffa0fe8801e800.idle_timeout=0 osc.lustre-OST0001-osc-ffffa0fe830c7800.idle_timeout=0 osc.lustre-OST0001-osc-ffffa0fe8801e800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.17845 s, 45.9 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.10074 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5167: 283079 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.04582 s, 89.4 kB/s osc.lustre-OST0000-osc-ffffa0fe830c7800.idle_timeout=debug osc.lustre-OST0000-osc-ffffa0fe8801e800.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0fe830c7800.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0fe8801e800.idle_timeout=debug PASS 94 (14s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 06:04:44 (1761127484) llite.lustre-ffffa0fe830c7800.fast_read=0 llite.lustre-ffffa0fe8801e800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.0144566 s, 850 kB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffffa0fe830c7800.fast_read=1 llite.lustre-ffffa0fe8801e800.fast_read=1 PASS 95b (13s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 06:04:56 (1761127496) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 06:04:59 (1761127499) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00718935 s, 570 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (6s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 06:05:05 (1761127505) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0937473 s, 22.4 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (6s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 06:05:11 (1761127511) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0897196 s, 23.4 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (6s) == sanityn test 100e: DoM: read on open and file size ==== 06:05:17 (1761127517) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00728346 s, 1.6 MB/s PASS 100e (6s) == sanityn test 101a: Discard DoM data on unlink ========= 06:05:23 (1761127523) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0012259 s, 3.3 MB/s PASS 101a (6s) == sanityn test 101b: Discard DoM data on rename ========= 06:05:29 (1761127529) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0118317 s, 346 kB/s PASS 101b (6s) == sanityn test 101c: Discard DoM data on close-unlink === 06:05:35 (1761127535) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00118687 s, 3.5 MB/s PASS 101c (7s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 06:05:44 (1761127544) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1761127546 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1761127549 Finishing test test23 at 1761127549 PASS 103 (12s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 06:05:56 (1761127556) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (3s) == sanityn test 105: Glimpse and lock cancel race ======== 06:05:59 (1761127559) fail_loc=0x416 PASS 105 (52s) == sanityn test 106a: Verify the btime via statx() ======= 06:06:51 (1761127611) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 06:06:54 (1761127614) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0141696 s, 72.3 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505473 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 06:06:55.000000000 -0400 Modify: 2025-10-22 06:06:55.000000000 -0400 Change: 2025-10-22 06:06:55.000000000 -0400 Birth: 2025-10-22 06:06:54.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505473 -rw-r--r-- 1 0 0 1761127614 1761127615 1761127615 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505473 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 06:06:55.000000000 -0400 Modify: 2025-10-22 06:06:55.000000000 -0400 Change: 2025-10-22 06:06:55.000000000 -0400 Birth: 2025-10-22 06:06:54.000000000 -0400 1761127615 1024 1 PASS 106b (6s) == sanityn test 106c: Verify statx attributes mask ======= 06:07:00 (1761127620) PASS 106c (5s) == sanityn test 107a: Basic grouplock conflict =========== 06:07:05 (1761127625) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.188832 s, 55.5 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2660 0xa64 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 First grouplock blocks second one PASS 107a (7s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 06:07:12 (1761127632) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.207837 s, 50.5 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2638 0xa4e 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7118 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (12s) == sanityn test 108a: lseek: parallel updates ============ 06:07:24 (1761127644) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0056755 s, 1.8 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.6951 s, 1.8 MB/s PASS 108a (10s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 06:07:34 (1761127654) 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: Force libcfs to create 2 CPU partitions oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 45 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: Force libcfs to create 2 CPU partitions oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 50 Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 192.168.201.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.201.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg143-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg143-server: detected 4 online CPUs by sysfs oleg143-server: Force libcfs to create 2 CPU partitions Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre Starting client: oleg143-client.virtnet: -o user_xattr,flock oleg143-server@tcp:/lustre /mnt/lustre2 PASS 109 (1137s) == sanityn test 110: do not grant another lock on resend ========================================================== 06:26:31 (1761128791) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 06:26:32.000000000 -0400 Modify: 2025-10-22 06:26:32.000000000 -0400 Change: 2025-10-22 06:26:32.000000000 -0400 Birth: 2025-10-22 06:26:32.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg143-client.virtnet oleg143-client.virtnet PASS 110 (42s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 06:27:13 (1761128833) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (3s) == sanityn test 112: update max-inherit in default LMV === 06:27:16 (1761128836) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (3s) == sanityn test 113: check servers of specified fs ======= 06:27:19 (1761128839) lustre-OST0000-osc-ffffa0fe84a60800 active. lustre-OST0001-osc-ffffa0fe84a60800 active. lustre-OST0000-osc-ffffa0fe84a60800 active. lustre-OST0001-osc-ffffa0fe84a60800 active. MGC192.168.201.143@tcp active. PASS 113 (7s) cleanup: ====================================================== == sanityn test complete, duration 7874 sec ============== 06:27:28 (1761128848) Stopping clients: oleg143-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg143-client.virtnet /mnt/lustre2 opts: