-----============= acceptance-small: sanityn ============----- Fri Jun 12 12:14:42 EDT 2026 oleg314-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Started clients oleg314-client.virtnet: 192.168.203.114@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg314-client.virtnet: executing check_config_client /mnt/lustre oleg314-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg314-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8dfe86913000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8dfe90fcc800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8dfe86913000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8dfe90fcc800.idle_timeout=debug disable quota as required oleg314-server: oleg314-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465147 s, 22.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7880] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 12:15:31 (1781280931) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (7s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 12:15:37 (1781280937) -rw-r--r-- 1 root root 0 Jun 12 12:15 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (7s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 12:15:44 (1781280944) -rw-r--r-- 1 root root 0 Jun 12 12:15 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (6s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 12:15:50 (1781280950) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (5s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 12:15:56 (1781280956) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (6s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 12:16:02 (1781280962) total 1 -rwxrwxrwx 1 root root 0 Jun 12 12:15 f2a -rwxrwxrwx 1 root root 0 Jun 12 12:15 f2b total 1 -rwxrwxrwx 1 root root 0 Jun 12 12:15 f2a -rwxrwxrwx 1 root root 0 Jun 12 12:15 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0186743 s, 27.4 kB/s PASS 2e (8s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 12:16:10 (1781280970) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (8s) == sanityn test 2g: check blocks update on sync write ==== 12:16:18 (1781280978) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.702872 s, 3.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (10s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 12:16:28 (1781280988) PASS 3 (8s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 12:16:36 (1781280996) PASS 4 (10s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 12:16:46 (1781281006) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (9s) == sanityn test 6: remove of open file on other node ============================================================================ 12:16:56 (1781281016) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (8s) == sanityn test 7: remove of open directory on other node ======================================================================= 12:17:04 (1781281024) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (7s) == sanityn test 8: remove of open special file on other node ==================================================================== 12:17:11 (1781281031) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (6s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 12:17:17 (1781281037) PASS 9a (8s) == sanityn test 9b: append to striped sparse file ======== 12:17:25 (1781281045) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775475 s, 13.5 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.062533 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (7s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 12:17:32 (1781281052) 1+0 records in 1+0 records out 1 byte copied, 0.00830324 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0620648 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.031292 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0152442 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0148437 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0300674 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0194576 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00540296 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00689509 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0076578 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0193059 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00521381 s, 0.2 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 12:17:39 (1781281059) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.027117 s, 113 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0553829 s, 74.0 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000245796 s, 12.5 MB/s PASS 10b (6s) == sanityn test 11: execution of file opened for write should return error ============================================================== 12:17:46 (1781281066) striped dir -i1 -c2 -H all_char /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (7s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 12:17:53 (1781281073) warning: '-runtime' deprecated, use '-t runtime' instead - link 338 (time 1781281087.13 total 10.03 last 33.71) - link 662 (time 1781281097.14 total 20.04 last 32.36) - link 971 (time 1781281107.15 total 30.04 last 30.88) - link 1318 (time 1781281117.17 total 40.07 last 34.62) - link 1641 (time 1781281127.18 total 50.07 last 32.28) - link 1939 (time 1781281137.20 total 60.10 last 29.73) - link 2258 (time 1781281147.23 total 70.13 last 31.81) - link 2567 (time 1781281157.25 total 80.14 last 30.85) - link 2884 (time 1781281167.28 total 90.18 last 31.59) total: 3182 link in 100.00 seconds: 31.82 ops/second using seed 1142736286 running for 100 seconds - stat 7743 (time 1781281089 ; total 11 ; last 11) - stat 10000 (time 1781281091 ; total 13 ; last 2) - stat 18601 (time 1781281102 ; total 24 ; last 11) - stat 20000 (time 1781281103 ; total 25 ; last 1) - stat 27950 (time 1781281114 ; total 36 ; last 11) - stat 30000 (time 1781281116 ; total 38 ; last 2) - stat 38148 (time 1781281127 ; total 49 ; last 11) - stat 40000 (time 1781281129 ; total 51 ; last 2) - stat 44587 (time 1781281140 ; total 62 ; last 11) - stat 49014 (time 1781281151 ; total 73 ; last 11) - stat 50000 (time 1781281153 ; total 75 ; last 2) - stat 54310 (time 1781281164 ; total 86 ; last 11) - stat 59032 (time 1781281175 ; total 97 ; last 11) - stat 60000 (time 1781281177 ; total 99 ; last 2) total: 60702 stats in 100 seconds: 607.020020 stats/second - unlinked 0 (time 1781281128 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile3182) error: No such file or directory total: 3181 unlinks in 82 seconds: 38.792683 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21010) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21012) - No such process PASS 12 (142s) == sanityn test 13: test directory page revocation ======= 12:20:16 (1781281216) striped dir -i1 -c2 -H crush /mnt/lustre/d13 f13 f13 PASS 13 (9s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 12:20:25 (1781281225) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22207 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (7s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 12:20:32 (1781281232) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22793 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (8s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 12:20:40 (1781281240) striped dir -i0 -c2 -H all_char /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23377 Terminated $DIR1/$tdir/sleep 60 PASS 14b (7s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 12:20:48 (1781281248) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23965 Terminated $DIR1/$tdir/sleep 60 PASS 14c (9s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 12:20:56 (1781281256) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24555 Terminated $DIR1/$tdir/sleep 60 PASS 14d (9s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 12:21:05 (1781281265) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (23s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 12:21:28 (1781281288) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.349672 s, 6.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.245255 s, 8.6 MB/s Chance of close/open is 1 in 50 Seed set to 1291 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1781281292.671098 mapwrite 0x5ab5ee thru 0x5b26a1 (0x70b4 bytes) 000003[0] 1781281292.964749 read 0x4219e5 thru 0x42395a (0x1f76 bytes) 000005[0] 1781281293.051958 write 0x40445c thru 0x408abd (0x4662 bytes) 000006[0] 1781281293.096118 mapwrite 0x0699e0 thru 0x06dac7 (0x40e8 bytes) 000007[1] 1781281293.192735 mapread 0x39b4e2 thru 0x3a7015 (0xbb34 bytes) 000008[0] 1781281293.263317 write 0x210b74 thru 0x218caa (0x8137 bytes) 000010[1] 1781281293.369269 mapread 0x2f5a98 thru 0x3024f7 (0xca60 bytes) 000011[1] 1781281293.458530 mapwrite 0x673012 thru 0x679a09 (0x69f8 bytes) truncating to largest ever: 0x4aef44 000012[1] 1781281293.717454 trunc from 0x679a0a to 0x4aef43 (0x1caac6 bytes) 000014[0] 1781281293.862662 mapread 0x4a9f04 thru 0x4aef43 (0x5040 bytes) 000015[0] 1781281293.925052 mapread 0x22bf36 thru 0x23ad13 (0xedde bytes) 000017[0] 1781281294.008405 write 0x68f129 thru 0x68f9e5 (0x8bd bytes) truncating to largest ever: 0x74d269 000020[0] 1781281294.092733 trunc from 0x68f9e6 to 0x74d268 (0xbd883 bytes) 000022[0] 1781281294.160356 read 0x3301bd thru 0x336113 (0x5f57 bytes) 000026[0] 1781281294.338440 mapread 0x632e95 thru 0x63c985 (0x9af1 bytes) 000028[0] 1781281294.362170 read 0x43bffd thru 0x43f104 (0x3108 bytes) 000030[1] 1781281294.383374 mapread 0x23274e thru 0x23ffb9 (0xd86c bytes) 000031[0] 1781281294.444020 trunc from 0x74d269 to 0x0629fd (0x6ea86b bytes) 000032[0] 1781281294.486177 mapwrite 0x2c5fb2 thru 0x2d3f4c (0xdf9b bytes) 000034[0] 1781281294.695087 write 0x76c5d7 thru 0x77721e (0xac48 bytes) 000035[0] 1781281294.729776 mapread 0x5d7ca3 thru 0x5e14b3 (0x9811 bytes) 000036[1] 1781281294.879255 read 0x7360c9 thru 0x74316b (0xd0a3 bytes) 000038[1] 1781281294.915772 write 0x051cde thru 0x057e36 (0x6159 bytes) 000039[1] 1781281294.940508 write 0x70d0c4 thru 0x7153ee (0x832b bytes) 000040[0] 1781281294.966845 mapread 0x14803a thru 0x1523c3 (0xa38a bytes) 000042[0] 1781281295.116470 mapwrite 0x2c40cc thru 0x2cc22e (0x8163 bytes) 000046[1] 1781281295.407750 read 0x70f94e thru 0x71f8a0 (0xff53 bytes) 000050[1] 1781281295.466011 write 0x55611b thru 0x56259a (0xc480 bytes) truncating to largest ever: 0x9e6393 000051[1] 1781281295.500525 trunc from 0x77721f to 0x9e6392 (0x26f174 bytes) 000052[1] 1781281295.594678 write 0x5df5a8 thru 0x5e3dbc (0x4815 bytes) 000055[1] 1781281295.729337 mapwrite 0x9d8e01 thru 0x9dbd66 (0x2f66 bytes) 000056[0] 1781281295.914834 mapread 0x2bf1bb thru 0x2c93f7 (0xa23d bytes) 000057[1] 1781281296.015890 mapwrite 0x3ff7b9 thru 0x40d7f2 (0xe03a bytes) 000060[0] 1781281296.302876 trunc from 0x9e6393 to 0x35845b (0x68df37 bytes) 000063[1] 1781281296.437389 trunc from 0x35845c to 0x8bc582 (0x564127 bytes) 000068[1] 1781281296.674268 write 0x3dd044 thru 0x3ebeaa (0xee67 bytes) 000069[0] 1781281296.736851 mapread 0x3e6382 thru 0x3f597a (0xf5f9 bytes) 000076[1] 1781281297.016497 mapread 0x076ac2 thru 0x084e06 (0xe345 bytes) 000078[0] 1781281297.088440 write 0x0e088a thru 0x0e3e93 (0x360a bytes) 000079[1] 1781281297.122371 trunc from 0x8bc583 to 0x903ada (0x47558 bytes) 000080[0] 1781281297.267409 write 0x385cb3 thru 0x392315 (0xc663 bytes) 000082[1] 1781281297.331674 write 0x49c438 thru 0x4a5115 (0x8cde bytes) 000083[0] 1781281297.385242 write 0x7ef043 thru 0x7ef613 (0x5d1 bytes) 000085[0] 1781281297.432812 mapread 0x581da8 thru 0x585252 (0x34ab bytes) 000087[1] 1781281297.523329 trunc from 0x903adb to 0x2b8fdd (0x64aafd bytes) 000088[0] 1781281297.583681 mapwrite 0x7ec0c6 thru 0x7f053d (0x4478 bytes) 000089[0] 1781281297.841848 mapwrite 0x138877 thru 0x13c3fe (0x3b88 bytes) 000094[0] 1781281297.971100 trunc from 0x7f053e to 0x7a4610 (0x4bf2d bytes) 000095[1] 1781281297.999030 mapwrite 0x01e8a8 thru 0x02cab3 (0xe20c bytes) 000096[1] 1781281298.318621 write 0x360804 thru 0x367984 (0x7181 bytes) 000098[0] 1781281298.410448 trunc from 0x7a4611 to 0x4a2fb6 (0x30165a bytes) 000099[0] 1781281298.506428 mapread 0x3df73d thru 0x3ea7fa (0xb0be bytes) 000100[1] 1781281298.542352 read 0x1cbce2 thru 0x1d5311 (0x9630 bytes) 000102[0] 1781281298.604123 read 0x179f58 thru 0x186c52 (0xccfb bytes) 000104[0] 1781281298.662378 trunc from 0x4a2fb7 to 0x105a52 (0x39d564 bytes) 000105[0] 1781281298.714403 write 0x4d2429 thru 0x4da610 (0x81e8 bytes) 000107[1] 1781281298.753698 trunc from 0x4da611 to 0x88af99 (0x3b0989 bytes) 000108[1] 1781281298.845157 mapwrite 0x2cc925 thru 0x2ceed6 (0x25b2 bytes) 000112[0] 1781281298.931868 mapread 0x699b48 thru 0x69a95f (0xe18 bytes) 000114[0] 1781281299.033474 mapwrite 0x631958 thru 0x6409bf (0xf068 bytes) 000116[0] 1781281299.273546 read 0x48b189 thru 0x497f4f (0xcdc7 bytes) 000119[1] 1781281299.370107 trunc from 0x88af9a to 0x2da59c (0x5b09fd bytes) 000120[0] 1781281299.450164 read 0x0c1155 thru 0x0ce9ed (0xd899 bytes) 000122[1] 1781281299.487420 trunc from 0x2da59d to 0x496bf6 (0x1bc65a bytes) 000126[1] 1781281299.694782 mapwrite 0x37d934 thru 0x38a3ea (0xcab7 bytes) 000129[1] 1781281299.876778 read 0x3b3d77 thru 0x3b865e (0x48e8 bytes) 000130[0] 1781281299.966609 write 0x413480 thru 0x4204b8 (0xd039 bytes) 000132[1] 1781281300.092167 mapread 0x21fe5c thru 0x22d8ea (0xda8f bytes) 000133[1] 1781281300.296680 read 0x30eea6 thru 0x3169ed (0x7b48 bytes) 000134[1] 1781281300.406953 mapwrite 0x20493d thru 0x2064d7 (0x1b9b bytes) 000135[0] 1781281300.570538 read 0x0e8590 thru 0x0ea53e (0x1faf bytes) 000138[0] 1781281300.672391 read 0x17c820 thru 0x17e875 (0x2056 bytes) 000140[1] 1781281300.820249 read 0x29e6dd thru 0x2a0e43 (0x2767 bytes) 000141[1] 1781281300.867127 mapread 0x21f61b thru 0x22ecae (0xf694 bytes) 000143[0] 1781281300.979656 mapread 0x0e5262 thru 0x0f04d9 (0xb278 bytes) 000144[0] 1781281300.992679 trunc from 0x496bf7 to 0x02320d (0x4739e9 bytes) 000145[0] 1781281301.054053 trunc from 0x02320e to 0x963bd0 (0x9409c3 bytes) 000149[1] 1781281301.131236 write 0x9a07a5 thru 0x9a74d0 (0x6d2c bytes) 000151[0] 1781281301.186393 read 0x8d4904 thru 0x8d87fe (0x3efb bytes) 000154[0] 1781281301.264456 trunc from 0x9a74d1 to 0x14728f (0x860241 bytes) 000156[0] 1781281301.373068 mapread 0x011ef0 thru 0x0157e3 (0x38f4 bytes) 000157[0] 1781281301.483082 mapwrite 0x1db808 thru 0x1ea7fa (0xeff3 bytes) 000160[0] 1781281301.906227 mapwrite 0x61a48f thru 0x6252b9 (0xae2b bytes) 000162[0] 1781281302.235357 write 0x41eaef thru 0x42168c (0x2b9e bytes) 000163[0] 1781281302.255934 read 0x509f7b thru 0x50f451 (0x54d7 bytes) 000166[0] 1781281302.362017 mapread 0x56704c thru 0x56b50c (0x44c1 bytes) 000169[0] 1781281302.444173 write 0x15c605 thru 0x169345 (0xcd41 bytes) 000170[0] 1781281302.495787 read 0x1ff56d thru 0x200e53 (0x18e7 bytes) 000171[0] 1781281302.588998 write 0x6831b4 thru 0x6918d3 (0xe720 bytes) 000172[1] 1781281302.602830 trunc from 0x6918d4 to 0x6c5771 (0x33e9e bytes) 000176[0] 1781281302.784087 write 0x0320b4 thru 0x03a533 (0x8480 bytes) 000177[1] 1781281302.841683 mapread 0x424b88 thru 0x42bd2a (0x71a3 bytes) 000182[1] 1781281302.956279 mapwrite 0x3e6242 thru 0x3f3d62 (0xdb21 bytes) 000186[1] 1781281303.154416 read 0x18cbec thru 0x194393 (0x77a8 bytes) 000187[0] 1781281303.251619 mapread 0x2cb512 thru 0x2d398b (0x847a bytes) 000192[0] 1781281303.393358 read 0x3a157c thru 0x3a2db8 (0x183d bytes) 000193[1] 1781281303.509946 trunc from 0x6c5772 to 0x67c079 (0x496f8 bytes) 000195[1] 1781281303.680847 mapwrite 0x10a1c2 thru 0x11108d (0x6ecc bytes) 000197[0] 1781281303.906599 mapread 0x64b8f7 thru 0x64f0fa (0x3804 bytes) 000198[0] 1781281304.007013 mapread 0x1b0f6b thru 0x1b4866 (0x38fc bytes) 000203[0] 1781281304.097410 mapwrite 0x9df6f3 thru 0x9ef154 (0xfa62 bytes) 000204[0] 1781281304.408460 mapwrite 0x244eb7 thru 0x24d386 (0x84d0 bytes) 000207[1] 1781281304.681017 mapwrite 0x3ab8f0 thru 0x3ba17d (0xe88e bytes) 000208[1] 1781281304.843349 trunc from 0x9ef155 to 0x95b22a (0x93f2a bytes) 000218[1] 1781281305.095326 read 0x45705f thru 0x457907 (0x8a9 bytes) 000219[1] 1781281305.281050 trunc from 0x95b22b to 0x9bbdd8 (0x60bae bytes) 000220[1] 1781281305.370607 trunc from 0x9bbdd9 to 0x690efd (0x32aedb bytes) 000221[0] 1781281305.432756 read 0x5e23ac thru 0x5f112d (0xed82 bytes) 000226[1] 1781281305.652516 read 0x4758ac thru 0x47b911 (0x6066 bytes) 000227[1] 1781281305.679394 mapread 0x608632 thru 0x60d557 (0x4f26 bytes) 000228[1] 1781281305.723980 mapwrite 0x43e056 thru 0x44aba2 (0xcb4d bytes) 000229[1] 1781281305.820886 write 0x211a1f thru 0x221863 (0xfe45 bytes) 000233[0] 1781281305.846408 mapread 0x2c70a1 thru 0x2cc783 (0x56e3 bytes) 000237[1] 1781281305.932675 trunc from 0x690efe to 0x38ad0b (0x3061f2 bytes) 000238[1] 1781281305.990638 mapread 0x20e010 thru 0x2179ca (0x99bb bytes) 000240[0] 1781281306.076792 mapread 0x2be8af thru 0x2c5370 (0x6ac2 bytes) 000242[1] 1781281306.199380 trunc from 0x38ad0c to 0x2b2894 (0xd8477 bytes) 000243[0] 1781281306.259443 write 0x0702f9 thru 0x0792b1 (0x8fb9 bytes) 000244[0] 1781281306.305295 write 0x9017d9 thru 0x910a37 (0xf25f bytes) 000246[0] 1781281306.339274 mapread 0x18f2ba thru 0x1981a0 (0x8ee7 bytes) 000247[1] 1781281306.439407 mapwrite 0x4c72f5 thru 0x4d6fa7 (0xfcb3 bytes) 000249[0] 1781281307.005692 trunc from 0x910a38 to 0x27d16e (0x6938c9 bytes) 000251[0] 1781281307.089073 write 0x6a37b3 thru 0x6b0b53 (0xd3a1 bytes) 000254[0] 1781281307.194612 write 0x253417 thru 0x25e206 (0xadf0 bytes) 000255[1] 1781281307.228380 mapread 0x06390a thru 0x071460 (0xdb57 bytes) 000257[1] 1781281307.408000 mapwrite 0x3d3cf4 thru 0x3d99f9 (0x5d06 bytes) 000261[0] 1781281307.573485 mapread 0x581191 thru 0x588585 (0x73f5 bytes) 000264[1] 1781281307.642786 read 0x1b8bbd thru 0x1c412b (0xb56f bytes) 000265[0] 1781281307.719762 mapwrite 0x7d8a9f thru 0x7dd36a (0x48cc bytes) 000266[1] 1781281307.960769 trunc from 0x7dd36b to 0x07b49f (0x761ecb bytes) 000272[0] 1781281308.085887 write 0x841b9f thru 0x843d76 (0x21d8 bytes) 000273[1] 1781281308.152396 mapwrite 0x6e6467 thru 0x6eb021 (0x4bbb bytes) 000275[0] 1781281308.326646 read 0x616a2b thru 0x62049a (0x9a70 bytes) 000276[0] 1781281308.412802 read 0x1638d3 thru 0x169e10 (0x653e bytes) 000279[0] 1781281308.512882 mapread 0x3474ef thru 0x35136c (0x9e7e bytes) 000280[1] 1781281308.518498 mapread 0x39dd68 thru 0x3a42a2 (0x653b bytes) 000281[1] 1781281308.586022 mapread 0x58909c thru 0x595620 (0xc585 bytes) 000282[0] 1781281308.691781 write 0x740081 thru 0x74be76 (0xbdf6 bytes) 000283[0] 1781281308.739992 mapread 0x32af65 thru 0x333fff (0x909b bytes) 000284[0] 1781281308.804671 write 0x3f512b thru 0x403ffd (0xeed3 bytes) 000287[1] 1781281308.893363 mapread 0x49fcad thru 0x4a813f (0x8493 bytes) 000289[0] 1781281309.055268 trunc from 0x843d77 to 0x1c3e66 (0x67ff10 bytes) 000290[1] 1781281309.112385 read 0x171e6c thru 0x17820f (0x63a4 bytes) 000291[1] 1781281309.155479 write 0x75126f thru 0x755f74 (0x4d06 bytes) 000292[0] 1781281309.196169 read 0x3d23a1 thru 0x3e012d (0xdd8d bytes) 000293[1] 1781281309.349769 mapread 0x321d73 thru 0x3315a3 (0xf831 bytes) 000294[1] 1781281309.516598 write 0x2ee5c5 thru 0x2f947f (0xaebb bytes) 000295[1] 1781281309.674446 mapwrite 0x5071fc thru 0x51703d (0xfe42 bytes) 000296[1] 1781281309.990771 read 0x71eda5 thru 0x72363a (0x4896 bytes) 000297[0] 1781281310.058060 write 0x58e16f thru 0x59bf97 (0xde29 bytes) 000298[0] 1781281310.103241 read 0x0ca833 thru 0x0d773f (0xcf0d bytes) 000301[0] 1781281310.359635 mapwrite 0x4ade4a thru 0x4b7ad0 (0x9c87 bytes) 000302[0] 1781281310.660222 write 0x038d66 thru 0x040c34 (0x7ecf bytes) 000306[0] 1781281310.715415 mapwrite 0x2f4f9d thru 0x2f6e6e (0x1ed2 bytes) 000307[0] 1781281310.843061 mapread 0x262cd6 thru 0x264633 (0x195e bytes) 000309[0] 1781281310.904587 mapread 0x66f24d thru 0x674fa9 (0x5d5d bytes) 000310[1] 1781281310.967593 read 0x4ae591 thru 0x4bdb63 (0xf5d3 bytes) 000314[1] 1781281311.129773 trunc from 0x755f75 to 0x20a13b (0x54be39 bytes) 000316[0] 1781281311.240417 mapread 0x0173e2 thru 0x017e4f (0xa6e bytes) 000317[1] 1781281311.275090 mapread 0x0dd9ba thru 0x0e67f1 (0x8e38 bytes) 000320[0] 1781281311.329065 read 0x0bf550 thru 0x0c4c5e (0x570f bytes) 000321[1] 1781281311.364508 write 0x382c8b thru 0x391de9 (0xf15f bytes) 000323[0] 1781281311.400513 trunc from 0x391dea to 0x1c77e6 (0x1ca603 bytes) 000325[0] 1781281311.511858 trunc from 0x1c77e7 to 0x7cb452 (0x603c6c bytes) 000326[0] 1781281311.556292 mapread 0x41ee91 thru 0x41f9a9 (0xb19 bytes) 000327[0] 1781281311.737458 write 0x91f1d3 thru 0x92c00c (0xce3a bytes) 000330[1] 1781281311.903219 read 0x853e62 thru 0x85cbf1 (0x8d90 bytes) 000331[1] 1781281312.009585 write 0x32d810 thru 0x3385b8 (0xada9 bytes) 000332[0] 1781281312.095374 mapread 0x69cc90 thru 0x6a66de (0x9a4f bytes) 000333[1] 1781281312.177859 write 0x8b8873 thru 0x8c6ba9 (0xe337 bytes) 000334[1] 1781281312.205454 mapread 0x179f12 thru 0x17fa20 (0x5b0f bytes) 000335[1] 1781281312.237193 read 0x5edef8 thru 0x5f2680 (0x4789 bytes) 000336[1] 1781281312.333630 trunc from 0x92c00d to 0x66f1fa (0x2bce12 bytes) 000338[1] 1781281312.359478 write 0x033bd1 thru 0x03fbf6 (0xc026 bytes) 000341[0] 1781281312.445615 trunc from 0x66f1fb to 0x71c110 (0xacf16 bytes) 000342[0] 1781281312.592150 trunc from 0x71c111 to 0x405efb (0x316215 bytes) 000343[0] 1781281312.644218 mapwrite 0x581779 thru 0x5869c2 (0x524a bytes) 000347[1] 1781281312.820323 mapread 0x11b612 thru 0x11ce6a (0x1859 bytes) 000350[0] 1781281312.874195 mapread 0x1a1911 thru 0x1a68f6 (0x4fe6 bytes) 000351[0] 1781281312.911528 write 0x15e67c thru 0x161f31 (0x38b6 bytes) 000354[1] 1781281312.964915 mapread 0x0f453c thru 0x0ffb59 (0xb61e bytes) 000356[0] 1781281313.059983 mapread 0x01b0d3 thru 0x02834e (0xd27c bytes) 000358[1] 1781281313.144197 mapwrite 0x72e39b thru 0x7345e5 (0x624b bytes) 000359[1] 1781281313.357218 write 0x93d431 thru 0x9425dd (0x51ad bytes) 000360[0] 1781281313.366304 mapread 0x238ad7 thru 0x2431e7 (0xa711 bytes) 000365[1] 1781281313.524862 write 0x28f004 thru 0x29c265 (0xd262 bytes) 000366[1] 1781281313.619614 mapread 0x64a9e1 thru 0x655e77 (0xb497 bytes) 000367[1] 1781281313.705602 trunc from 0x9425de to 0x9d3b6b (0x9158e bytes) 000369[0] 1781281313.781743 mapwrite 0x44d573 thru 0x4586e4 (0xb172 bytes) 000371[0] 1781281314.105208 mapread 0x6ea5e6 thru 0x6f4d8b (0xa7a6 bytes) 000372[0] 1781281314.288373 write 0x4cbcad thru 0x4d96a5 (0xd9f9 bytes) 000373[0] 1781281314.328457 trunc from 0x9d3b6c to 0x77db12 (0x256059 bytes) 000374[0] 1781281314.386719 read 0x656d03 thru 0x65c3a6 (0x56a4 bytes) 000376[1] 1781281314.488034 write 0x9517ab thru 0x952645 (0xe9b bytes) 000379[1] 1781281314.551042 mapread 0x4c6745 thru 0x4c706b (0x927 bytes) 000380[1] 1781281314.629070 read 0x48f736 thru 0x491f79 (0x2844 bytes) 000381[0] 1781281314.741532 trunc from 0x952646 to 0x84562b (0x10d01a bytes) 000384[1] 1781281314.858350 read 0x2a7154 thru 0x2b35ff (0xc4ac bytes) 000385[0] 1781281314.915445 trunc from 0x84562c to 0x12a611 (0x71b01a bytes) 000389[1] 1781281315.023149 read 0x0c5a3a thru 0x0cbfbe (0x6585 bytes) 000390[1] 1781281315.071622 trunc from 0x12a612 to 0x45eee5 (0x3348d4 bytes) 000391[1] 1781281315.205656 trunc from 0x45eee6 to 0x1ea6b3 (0x274832 bytes) 000392[0] 1781281315.258126 read 0x0f61dd thru 0x105bc9 (0xf9ed bytes) 000395[0] 1781281315.374519 trunc from 0x1ea6b4 to 0x3e6ce8 (0x1fc635 bytes) 000397[0] 1781281315.567514 mapread 0x1d1f7a thru 0x1db171 (0x91f8 bytes) 000399[0] 1781281315.748302 write 0x00b6b7 thru 0x0130d2 (0x7a1c bytes) 000400[1] 1781281315.783107 mapread 0x1196fb thru 0x1254cd (0xbdd3 bytes) 000401[1] 1781281315.844965 mapwrite 0x8ae4e5 thru 0x8bd0ae (0xebca bytes) 000403[1] 1781281316.149325 trunc from 0x8bd0af to 0x98f841 (0xd2793 bytes) 000407[0] 1781281316.232521 trunc from 0x98f842 to 0x271c9e (0x71dba3 bytes) 000412[0] 1781281316.378942 trunc from 0x271c9f to 0x7f6ee1 (0x585243 bytes) 000413[0] 1781281316.434867 trunc from 0x7f6ee2 to 0x14a9e5 (0x6ac4fc bytes) 000416[1] 1781281316.493493 mapread 0x0ad082 thru 0x0b1901 (0x4880 bytes) 000417[0] 1781281316.585189 mapwrite 0x82f7ad thru 0x8312bc (0x1b10 bytes) 000418[1] 1781281316.800425 mapread 0x7866e0 thru 0x791532 (0xae53 bytes) 000420[1] 1781281316.976351 write 0x310728 thru 0x314397 (0x3c70 bytes) 000422[1] 1781281317.106806 trunc from 0x8312bd to 0x00c4b9 (0x824e03 bytes) 000423[1] 1781281317.184803 write 0x2f1e80 thru 0x301095 (0xf216 bytes) 000425[1] 1781281317.301860 mapread 0x2a6351 thru 0x2aff32 (0x9be2 bytes) 000429[0] 1781281317.476102 mapwrite 0x81e4b8 thru 0x8271a8 (0x8cf1 bytes) 000430[0] 1781281317.767284 mapread 0x1a9778 thru 0x1b1c3a (0x84c3 bytes) 000433[0] 1781281317.911148 write 0x0f2101 thru 0x1005c3 (0xe4c3 bytes) 000435[1] 1781281317.957509 read 0x3d8309 thru 0x3d9623 (0x131b bytes) 000436[0] 1781281318.023848 mapread 0x34b6ac thru 0x353185 (0x7ada bytes) 000437[1] 1781281318.067226 write 0x171334 thru 0x17dee6 (0xcbb3 bytes) 000438[0] 1781281318.109386 trunc from 0x8271a9 to 0x255dad (0x5d13fb bytes) 000439[0] 1781281318.251654 write 0x230b95 thru 0x23e39e (0xd80a bytes) 000440[0] 1781281318.276237 write 0x80efd1 thru 0x81cde5 (0xde15 bytes) 000442[1] 1781281318.317441 mapwrite 0x420a8b thru 0x42edab (0xe321 bytes) 000444[1] 1781281318.563163 write 0x7efdf1 thru 0x7fe181 (0xe391 bytes) 000446[0] 1781281318.683421 mapwrite 0x7e55bb thru 0x7eb2e9 (0x5d2f bytes) 000447[1] 1781281318.895326 trunc from 0x81cde6 to 0x26411e (0x5b8cc7 bytes) 000449[1] 1781281319.032701 trunc from 0x26411f to 0x23a2ee (0x29e30 bytes) 000450[0] 1781281319.092028 mapwrite 0x0811b8 thru 0x0840d6 (0x2f1f bytes) 000451[1] 1781281319.219974 write 0x5a7f16 thru 0x5b277a (0xa865 bytes) 000455[1] 1781281319.294962 read 0x5a5cf8 thru 0x5adc8f (0x7f98 bytes) 000456[0] 1781281319.318035 write 0x81226a thru 0x81dbf8 (0xb98f bytes) 000457[0] 1781281319.336024 mapwrite 0x0f9e32 thru 0x1034a6 (0x9675 bytes) 000462[1] 1781281319.587432 read 0x654d1f thru 0x6644f0 (0xf7d2 bytes) 000463[0] 1781281319.657058 read 0x4e2966 thru 0x4e2fd0 (0x66b bytes) 000464[0] 1781281319.764283 read 0x6db019 thru 0x6ea946 (0xf92e bytes) 000466[0] 1781281319.797427 mapwrite 0x9f07a3 thru 0x9fffab (0xf809 bytes) 000467[0] 1781281320.118966 read 0x1e1558 thru 0x1e8dac (0x7855 bytes) 000468[1] 1781281320.272801 write 0x4622f1 thru 0x46ddc9 (0xbad9 bytes) 000469[1] 1781281320.312403 mapwrite 0x8f1412 thru 0x8fce96 (0xba85 bytes) 000470[1] 1781281320.459020 trunc from 0x9fffac to 0x5d6b66 (0x429445 bytes) 000472[1] 1781281320.564807 trunc from 0x5d6b67 to 0x8e94da (0x312974 bytes) 000473[1] 1781281320.603311 mapread 0x310103 thru 0x31c9cd (0xc8cb bytes) 000474[0] 1781281320.778055 mapread 0x27acd7 thru 0x27edb9 (0x40e3 bytes) 000477[1] 1781281320.855469 read 0x27b83b thru 0x28ac59 (0xf41f bytes) 000479[0] 1781281321.013089 write 0x086173 thru 0x089925 (0x37b3 bytes) 000480[0] 1781281321.066613 read 0x2e6778 thru 0x2f4ae9 (0xe372 bytes) 000481[0] 1781281321.096820 mapread 0x23a389 thru 0x249cc3 (0xf93b bytes) 000483[1] 1781281321.239846 write 0x038ebb thru 0x03b85f (0x29a5 bytes) 000484[1] 1781281321.276406 mapwrite 0x8673ef thru 0x86b9be (0x45d0 bytes) 000485[0] 1781281321.496133 mapwrite 0x26756f thru 0x2692c8 (0x1d5a bytes) 000486[1] 1781281321.643498 write 0x6f4f74 thru 0x6fc088 (0x7115 bytes) 000488[0] 1781281321.671153 write 0x68b87f thru 0x68ba19 (0x19b bytes) 000489[1] 1781281321.710901 mapwrite 0x26368f thru 0x268dd1 (0x5743 bytes) 000490[1] 1781281321.843190 mapread 0x45f33f thru 0x461721 (0x23e3 bytes) 000491[1] 1781281321.883074 mapread 0x54bccf thru 0x54be29 (0x15b bytes) 000492[0] 1781281321.883794 read 0x30f46c thru 0x31aa61 (0xb5f6 bytes) 000498[1] 1781281321.991258 trunc from 0x8e94db to 0x34ffb8 (0x599522 bytes) 000499[1] 1781281322.053307 read 0x1c8bb9 thru 0x1d0b3f (0x7f87 bytes) 000500[1] 1781281322.102119 read 0x0f57d3 thru 0x0fb431 (0x5c5f bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1322 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000003[1] 1781281322.979427 write_OD 0x8f3000 thru 0x8f8fff (0x6000 bytes) 000004[0] 1781281323.081001 write_OD 0x077000 thru 0x07ffff (0x9000 bytes) 000005[0] 1781281323.169693 read_OD 0x51a000 thru 0x51ffff (0x6000 bytes) 000007[0] 1781281323.182331 mapwrite 0x26f000 thru 0x274712 (0x5713 bytes) truncating to largest ever: 0x4d972d 000009[1] 1781281323.351778 trunc from 0x8f9000 to 0x4d972c (0x41f8d3 bytes) 000014[1] 1781281323.466314 mapwrite 0x4e7000 thru 0x4e9a77 (0x2a78 bytes) 000015[1] 1781281323.589136 read_OD 0x412000 thru 0x416fff (0x5000 bytes) 000016[1] 1781281323.620908 mapwrite 0x91c000 thru 0x92696c (0xa96d bytes) 000020[1] 1781281323.834162 mapread 0x599000 thru 0x5a0859 (0x785a bytes) 000022[1] 1781281323.866196 mapwrite 0x25e000 thru 0x261043 (0x3044 bytes) 000025[0] 1781281323.988229 read_OD 0x538000 thru 0x544fff (0xd000 bytes) 000026[1] 1781281324.023415 read_OD 0x76a000 thru 0x774fff (0xb000 bytes) 000027[0] 1781281324.041488 trunc from 0x92696d to 0x42bf34 (0x4faa38 bytes) 000028[1] 1781281324.130744 mapwrite 0x53a000 thru 0x53ae59 (0xe5a bytes) 000030[1] 1781281324.250033 read_OD 0x04a000 thru 0x051fff (0x8000 bytes) 000033[1] 1781281324.273155 mapwrite 0x086000 thru 0x08fad0 (0x9ad1 bytes) 000034[1] 1781281324.554017 read_OD 0x1a2000 thru 0x1adfff (0xc000 bytes) 000035[0] 1781281324.626801 write_OD 0x5b1000 thru 0x5b7fff (0x7000 bytes) 000036[0] 1781281324.754620 mapread 0x0ad000 thru 0x0ad883 (0x884 bytes) 000041[1] 1781281324.789900 trunc from 0x5b8000 to 0x4d5594 (0xe2a6b bytes) 000042[0] 1781281324.850284 trunc from 0x4d5595 to 0x189428 (0x34c16c bytes) 000044[1] 1781281324.890195 write_OD 0x015000 thru 0x019fff (0x5000 bytes) 000045[0] 1781281324.938052 mapwrite 0x3c1000 thru 0x3c986f (0x8870 bytes) 000046[1] 1781281325.205956 trunc from 0x3c9870 to 0x282719 (0x147156 bytes) 000047[0] 1781281325.268476 mapread 0x1c2000 thru 0x1cd73e (0xb73f bytes) 000049[0] 1781281325.383459 mapwrite 0x6d2000 thru 0x6deb49 (0xcb4a bytes) 000050[1] 1781281325.558293 mapwrite 0x563000 thru 0x568dc5 (0x5dc6 bytes) 000055[1] 1781281325.845598 mapread 0x483000 thru 0x4860cd (0x30ce bytes) 000056[0] 1781281325.846443 mapwrite 0x2d2000 thru 0x2dd455 (0xb456 bytes) 000057[0] 1781281326.010301 trunc from 0x6deb4a to 0x2546b0 (0x48a499 bytes) 000058[0] 1781281326.069136 read_OD 0x012000 thru 0x018fff (0x7000 bytes) 000059[1] 1781281326.098873 mapread 0x18b000 thru 0x191e86 (0x6e87 bytes) 000060[1] 1781281326.148894 mapwrite 0x5be000 thru 0x5c1275 (0x3276 bytes) 000063[1] 1781281326.310240 read_OD 0x433000 thru 0x43dfff (0xb000 bytes) 000064[1] 1781281326.329197 mapwrite 0x2a2000 thru 0x2a6e1d (0x4e1e bytes) 000066[0] 1781281326.486862 read_OD 0x22c000 thru 0x231fff (0x6000 bytes) 000067[1] 1781281326.516929 mapread 0x452000 thru 0x456e77 (0x4e78 bytes) 000068[0] 1781281326.551853 mapread 0x4ed000 thru 0x4f3c5e (0x6c5f bytes) 000070[0] 1781281326.610819 mapread 0x192000 thru 0x199f35 (0x7f36 bytes) 000072[0] 1781281326.648109 trunc from 0x5c1276 to 0x26d022 (0x354253 bytes) 000073[0] 1781281326.695207 write_OD 0x161000 thru 0x168fff (0x8000 bytes) 000077[1] 1781281326.792028 mapwrite 0x935000 thru 0x9391ea (0x41eb bytes) truncating to largest ever: 0x55c84d 000078[0] 1781281327.028115 trunc from 0x9391eb to 0x55c84c (0x3dc99e bytes) 000079[1] 1781281327.087752 mapwrite 0x8f5000 thru 0x8fbb15 (0x6b16 bytes) 000081[1] 1781281327.345769 read_OD 0x415000 thru 0x415fff (0x1000 bytes) 000082[1] 1781281327.392956 mapread 0x133000 thru 0x13b1eb (0x81ec bytes) 000083[1] 1781281327.614338 trunc from 0x8fbb16 to 0x510518 (0x3eb5fd bytes) 000087[0] 1781281327.760774 mapread 0x047000 thru 0x04dab1 (0x6ab2 bytes) 000088[1] 1781281327.839156 read_OD 0x08a000 thru 0x098fff (0xf000 bytes) 000090[1] 1781281327.847778 read_OD 0x1ba000 thru 0x1bbfff (0x2000 bytes) 000091[0] 1781281327.853339 read_OD 0x070000 thru 0x076fff (0x7000 bytes) 000093[0] 1781281327.861793 write_OD 0x455000 thru 0x45dfff (0x9000 bytes) 000096[1] 1781281327.934135 mapwrite 0x2f2000 thru 0x2f8aea (0x6aeb bytes) 000097[1] 1781281328.049710 mapwrite 0x3da000 thru 0x3e728d (0xd28e bytes) 000099[1] 1781281328.105884 mapwrite 0x2b8000 thru 0x2c4947 (0xc948 bytes) 000101[0] 1781281328.159744 read_OD 0x45f000 thru 0x463fff (0x5000 bytes) 000102[1] 1781281328.193182 write_OD 0x43a000 thru 0x445fff (0xc000 bytes) 000103[0] 1781281328.236113 mapread 0x03f000 thru 0x04cb04 (0xdb05 bytes) 000104[0] 1781281328.312369 read_OD 0x18a000 thru 0x191fff (0x8000 bytes) 000105[0] 1781281328.321034 write_OD 0x2f6000 thru 0x300fff (0xb000 bytes) 000107[0] 1781281328.367756 write_OD 0x313000 thru 0x31bfff (0x9000 bytes) 000111[0] 1781281328.439069 trunc from 0x510519 to 0x3bf4d9 (0x15103f bytes) 000115[0] 1781281328.512867 mapread 0x211000 thru 0x218937 (0x7938 bytes) 000118[0] 1781281328.616895 read_OD 0x353000 thru 0x361fff (0xf000 bytes) 000120[1] 1781281328.643144 write_OD 0x429000 thru 0x437fff (0xf000 bytes) 000121[1] 1781281328.694932 mapread 0x174000 thru 0x1795d8 (0x55d9 bytes) 000122[0] 1781281328.722220 mapread 0x31e000 thru 0x3278fd (0x98fe bytes) 000123[1] 1781281328.803757 write_OD 0x6a9000 thru 0x6acfff (0x4000 bytes) 000124[1] 1781281328.863057 write_OD 0x72e000 thru 0x735fff (0x8000 bytes) 000125[1] 1781281328.918004 mapwrite 0x5df000 thru 0x5e8121 (0x9122 bytes) truncating to largest ever: 0x8e82a2 000129[1] 1781281329.105531 trunc from 0x736000 to 0x8e82a1 (0x1b22a2 bytes) 000132[0] 1781281329.151874 read_OD 0x1c3000 thru 0x1c4fff (0x2000 bytes) 000136[1] 1781281329.175085 read_OD 0x44d000 thru 0x44efff (0x2000 bytes) 000141[1] 1781281329.182861 mapread 0x5ef000 thru 0x5f35c5 (0x45c6 bytes) 000142[0] 1781281329.288816 read_OD 0x5cb000 thru 0x5ccfff (0x2000 bytes) 000143[0] 1781281329.294182 write_OD 0x8a9000 thru 0x8b4fff (0xc000 bytes) 000144[1] 1781281329.367367 read_OD 0x00e000 thru 0x019fff (0xc000 bytes) 000145[0] 1781281329.391311 mapread 0x446000 thru 0x44d0ff (0x7100 bytes) 000146[0] 1781281329.544850 mapwrite 0x401000 thru 0x409e42 (0x8e43 bytes) 000149[0] 1781281329.717065 write_OD 0x66c000 thru 0x677fff (0xc000 bytes) 000152[1] 1781281329.817012 mapwrite 0x533000 thru 0x53456e (0x156f bytes) 000153[0] 1781281330.125006 write_OD 0x470000 thru 0x476fff (0x7000 bytes) 000154[1] 1781281330.189070 trunc from 0x8e82a2 to 0x04311e (0x8a5183 bytes) 000156[0] 1781281330.291171 trunc from 0x04311f to 0x7be31b (0x77b1fd bytes) 000157[0] 1781281330.317253 trunc from 0x7be31c to 0x26178d (0x55cb8e bytes) 000158[1] 1781281330.330203 read_OD 0x0d9000 thru 0x0e4fff (0xc000 bytes) 000160[0] 1781281330.391024 trunc from 0x26178e to 0x2cba03 (0x6a276 bytes) 000161[1] 1781281330.435607 trunc from 0x2cba04 to 0x33de4b (0x72448 bytes) 000162[1] 1781281330.465849 mapwrite 0x1a7000 thru 0x1b02b4 (0x92b5 bytes) 000165[1] 1781281330.731807 mapwrite 0x96c000 thru 0x973c4e (0x7c4f bytes) 000166[1] 1781281330.949632 trunc from 0x973c4f to 0x5f6e10 (0x37ce3e bytes) 000168[0] 1781281331.008818 mapwrite 0x6ae000 thru 0x6b719d (0x919e bytes) 000171[1] 1781281331.214111 write_OD 0x7e4000 thru 0x7e5fff (0x2000 bytes) 000172[0] 1781281331.270621 mapread 0x320000 thru 0x32911c (0x911d bytes) 000173[0] 1781281331.348525 trunc from 0x7e6000 to 0x480063 (0x365f9c bytes) 000174[0] 1781281331.408083 write_OD 0x9f8000 thru 0x9fefff (0x7000 bytes) 000177[1] 1781281331.495292 trunc from 0x9ff000 to 0x745246 (0x2b9db9 bytes) 000178[0] 1781281331.560834 write_OD 0x8a9000 thru 0x8adfff (0x5000 bytes) truncating to largest ever: 0x9aac1a 000182[1] 1781281331.654627 trunc from 0x8ae000 to 0x9aac19 (0xfcc1a bytes) 000183[1] 1781281331.702867 trunc from 0x9aac1a to 0x8651a3 (0x145a76 bytes) 000185[0] 1781281331.775786 read_OD 0x29c000 thru 0x29dfff (0x2000 bytes) 000186[0] 1781281331.800733 mapread 0x6fa000 thru 0x702727 (0x8728 bytes) 000189[1] 1781281331.929925 mapread 0x2be000 thru 0x2cd791 (0xf792 bytes) 000191[0] 1781281332.057154 mapwrite 0x885000 thru 0x8887f7 (0x37f8 bytes) 000192[0] 1781281332.227958 trunc from 0x8887f8 to 0x4a3015 (0x3e57e2 bytes) 000194[1] 1781281332.246136 mapwrite 0x192000 thru 0x1a0a12 (0xea13 bytes) 000199[0] 1781281332.483928 write_OD 0x674000 thru 0x67dfff (0xa000 bytes) 000200[0] 1781281332.541172 trunc from 0x67e000 to 0x937f6c (0x2b9f6d bytes) 000201[1] 1781281332.592229 mapread 0x26e000 thru 0x274f3d (0x6f3e bytes) 000202[1] 1781281332.667440 mapwrite 0x927000 thru 0x931c39 (0xac3a bytes) 000204[0] 1781281333.006497 mapread 0x8f3000 thru 0x8f9146 (0x6147 bytes) 000206[0] 1781281333.071544 mapwrite 0x08e000 thru 0x08fd9f (0x1da0 bytes) 000207[1] 1781281333.176945 write_OD 0x4dc000 thru 0x4e1fff (0x6000 bytes) 000208[0] 1781281333.340028 trunc from 0x937f6d to 0x214e59 (0x723113 bytes) 000209[0] 1781281333.378897 mapwrite 0x97c000 thru 0x98adfe (0xedff bytes) 000210[0] 1781281333.600653 mapwrite 0x265000 thru 0x269848 (0x4849 bytes) 000212[0] 1781281333.735692 read_OD 0x435000 thru 0x43dfff (0x9000 bytes) 000214[0] 1781281333.763122 read_OD 0x948000 thru 0x94ffff (0x8000 bytes) 000217[0] 1781281333.774987 mapread 0x140000 thru 0x148d11 (0x8d12 bytes) 000219[0] 1781281333.824051 mapwrite 0x072000 thru 0x07d936 (0xb937 bytes) 000220[0] 1781281333.945145 mapread 0x51c000 thru 0x5251a4 (0x91a5 bytes) 000222[1] 1781281334.110597 write_OD 0x3da000 thru 0x3e1fff (0x8000 bytes) 000223[1] 1781281334.239134 read_OD 0x6a7000 thru 0x6b4fff (0xe000 bytes) 000224[1] 1781281334.261329 mapread 0x5ee000 thru 0x5f6dd1 (0x8dd2 bytes) 000226[1] 1781281334.380351 write_OD 0x91a000 thru 0x91ffff (0x6000 bytes) 000227[1] 1781281334.435377 read_OD 0x5fa000 thru 0x5fdfff (0x4000 bytes) 000228[1] 1781281334.443461 mapwrite 0x33b000 thru 0x347713 (0xc714 bytes) 000231[0] 1781281334.747294 mapread 0x1d5000 thru 0x1e4a70 (0xfa71 bytes) 000232[1] 1781281334.882787 mapread 0x123000 thru 0x12c965 (0x9966 bytes) 000237[1] 1781281334.930998 read_OD 0x6b1000 thru 0x6b8fff (0x8000 bytes) 000238[0] 1781281334.939032 read_OD 0x624000 thru 0x632fff (0xf000 bytes) 000239[0] 1781281334.947260 write_OD 0x18a000 thru 0x198fff (0xf000 bytes) 000241[0] 1781281334.995287 mapwrite 0x253000 thru 0x258e8e (0x5e8f bytes) 000242[0] 1781281335.222760 mapwrite 0x6a1000 thru 0x6ae44c (0xd44d bytes) 000243[1] 1781281335.335474 mapread 0x1f9000 thru 0x1fd61f (0x4620 bytes) 000244[0] 1781281335.473692 mapwrite 0x393000 thru 0x396709 (0x370a bytes) 000246[1] 1781281335.591101 read_OD 0x30e000 thru 0x311fff (0x4000 bytes) 000247[1] 1781281335.625617 mapwrite 0x371000 thru 0x3767a0 (0x57a1 bytes) 000248[1] 1781281335.864152 mapread 0x52f000 thru 0x53da8b (0xea8c bytes) 000249[1] 1781281335.885029 mapread 0x76b000 thru 0x77173b (0x673c bytes) 000250[1] 1781281335.886469 write_OD 0x1ab000 thru 0x1b1fff (0x7000 bytes) 000251[0] 1781281335.955875 mapwrite 0x788000 thru 0x790906 (0x8907 bytes) 000255[0] 1781281336.102187 mapread 0x2ac000 thru 0x2ad049 (0x104a bytes) 000257[1] 1781281336.132041 mapread 0x2fd000 thru 0x30364f (0x6650 bytes) 000259[0] 1781281336.191872 mapread 0x4e4000 thru 0x4ec478 (0x8479 bytes) 000260[0] 1781281336.241172 read_OD 0x76e000 thru 0x775fff (0x8000 bytes) 000261[1] 1781281336.249674 read_OD 0x1e3000 thru 0x1e7fff (0x5000 bytes) 000264[0] 1781281336.260003 mapwrite 0x813000 thru 0x8200c3 (0xd0c4 bytes) 000265[1] 1781281336.394246 write_OD 0x20a000 thru 0x20bfff (0x2000 bytes) 000266[0] 1781281336.442341 trunc from 0x98adff to 0x7aeff6 (0x1dbe08 bytes) 000267[1] 1781281336.489372 mapread 0x43b000 thru 0x4470a9 (0xc0aa bytes) 000268[0] 1781281336.607028 write_OD 0x4f7000 thru 0x4f7fff (0x1000 bytes) 000270[1] 1781281336.678380 read_OD 0x05f000 thru 0x064fff (0x6000 bytes) 000274[1] 1781281336.687363 write_OD 0x779000 thru 0x786fff (0xe000 bytes) 000275[0] 1781281336.738023 trunc from 0x7aeff7 to 0x3d75c9 (0x3d7a2d bytes) 000278[0] 1781281336.795213 write_OD 0x32f000 thru 0x33bfff (0xd000 bytes) 000279[1] 1781281336.846687 trunc from 0x3d75ca to 0x0ea2d5 (0x2ed2f4 bytes) 000280[0] 1781281336.896803 mapread 0x019000 thru 0x020e0c (0x7e0d bytes) 000281[1] 1781281336.959979 mapwrite 0x9d4000 thru 0x9da057 (0x6058 bytes) 000282[1] 1781281337.268959 mapread 0x470000 thru 0x4790ba (0x90bb bytes) 000283[0] 1781281337.305222 mapread 0x891000 thru 0x895793 (0x4794 bytes) 000284[1] 1781281337.402996 mapwrite 0x651000 thru 0x65b2f3 (0xa2f4 bytes) 000286[0] 1781281337.570150 trunc from 0x9da058 to 0x157389 (0x882cce bytes) 000291[1] 1781281337.709410 read_OD 0x0f1000 thru 0x0fffff (0xf000 bytes) 000293[0] 1781281337.767237 read_OD 0x080000 thru 0x08bfff (0xc000 bytes) 000294[0] 1781281337.818468 mapread 0x155000 thru 0x156a23 (0x1a24 bytes) 000296[0] 1781281337.868387 read_OD 0x085000 thru 0x08cfff (0x8000 bytes) 000297[0] 1781281337.875722 read_OD 0x0d1000 thru 0x0dafff (0xa000 bytes) 000298[0] 1781281337.882917 trunc from 0x15738a to 0x3615d7 (0x20a24e bytes) 000300[0] 1781281337.919004 read_OD 0x0e8000 thru 0x0f5fff (0xe000 bytes) 000302[0] 1781281337.942825 trunc from 0x3615d8 to 0x3b8766 (0x5718f bytes) 000304[1] 1781281337.994980 write_OD 0x249000 thru 0x24ffff (0x7000 bytes) 000306[1] 1781281338.045008 mapread 0x2a7000 thru 0x2b3758 (0xc759 bytes) 000308[0] 1781281338.103120 write_OD 0x757000 thru 0x757fff (0x1000 bytes) 000309[1] 1781281338.174384 write_OD 0x844000 thru 0x84ffff (0xc000 bytes) 000310[1] 1781281338.269020 mapwrite 0x1b1000 thru 0x1b84b2 (0x74b3 bytes) 000315[1] 1781281338.450900 trunc from 0x850000 to 0x66b95f (0x1e46a0 bytes) 000318[1] 1781281338.494279 read_OD 0x4ee000 thru 0x4fcfff (0xf000 bytes) 000320[1] 1781281338.521715 mapread 0x3e4000 thru 0x3ee964 (0xa965 bytes) 000322[0] 1781281338.655080 read_OD 0x0f9000 thru 0x103fff (0xb000 bytes) 000324[1] 1781281338.669422 mapwrite 0x3be000 thru 0x3c76c0 (0x96c1 bytes) 000325[1] 1781281338.823391 trunc from 0x66b960 to 0x1c494d (0x4a7012 bytes) 000328[1] 1781281338.879450 trunc from 0x1c494e to 0x6c5188 (0x50083b bytes) 000329[0] 1781281338.903847 read_OD 0x554000 thru 0x55cfff (0x9000 bytes) 000331[0] 1781281338.933893 mapread 0x52e000 thru 0x536c45 (0x8c46 bytes) 000332[0] 1781281339.063184 trunc from 0x6c5189 to 0x0a0309 (0x624e7f bytes) 000336[0] 1781281339.186365 mapread 0x02e000 thru 0x02fb8a (0x1b8b bytes) 000338[1] 1781281339.200007 mapread 0x048000 thru 0x057f0e (0xff0f bytes) 000339[0] 1781281339.410474 mapwrite 0x869000 thru 0x8732d5 (0xa2d6 bytes) 000340[0] 1781281339.583897 mapwrite 0x0e9000 thru 0x0f24ff (0x9500 bytes) 000341[0] 1781281339.795083 write_OD 0x042000 thru 0x048fff (0x7000 bytes) 000343[1] 1781281339.950919 mapwrite 0x75a000 thru 0x75c08f (0x2090 bytes) 000344[1] 1781281340.093527 trunc from 0x8732d6 to 0x217ee0 (0x65b3f5 bytes) 000346[0] 1781281340.149232 write_OD 0x7c8000 thru 0x7cdfff (0x6000 bytes) 000347[1] 1781281340.241182 mapread 0x00f000 thru 0x016348 (0x7349 bytes) 000349[1] 1781281340.327536 trunc from 0x7ce000 to 0x5ab72f (0x2228d0 bytes) 000350[1] 1781281340.375125 mapwrite 0x5ae000 thru 0x5bab73 (0xcb74 bytes) 000351[1] 1781281340.593213 mapread 0x2ae000 thru 0x2bc583 (0xe584 bytes) 000352[0] 1781281340.714186 read_OD 0x217000 thru 0x21cfff (0x6000 bytes) 000354[1] 1781281340.738985 mapwrite 0x211000 thru 0x21b3d7 (0xa3d8 bytes) truncating to largest ever: 0x9e0909 000355[0] 1781281340.962124 trunc from 0x5bab74 to 0x9e0908 (0x425d95 bytes) 000356[0] 1781281341.023234 write_OD 0x351000 thru 0x352fff (0x2000 bytes) 000357[1] 1781281341.147940 read_OD 0x555000 thru 0x557fff (0x3000 bytes) 000360[0] 1781281341.174890 trunc from 0x9e0909 to 0x0fd74e (0x8e31ba bytes) 000361[1] 1781281341.263978 write_OD 0x9b4000 thru 0x9b6fff (0x3000 bytes) 000364[0] 1781281341.366334 write_OD 0x201000 thru 0x208fff (0x8000 bytes) 000367[0] 1781281341.455235 mapwrite 0x02b000 thru 0x032aae (0x7aaf bytes) 000368[0] 1781281341.717256 mapwrite 0x9d1000 thru 0x9e0be8 (0xfbe9 bytes) 000369[1] 1781281341.848482 mapread 0x100000 thru 0x107d9b (0x7d9c bytes) 000370[0] 1781281342.110517 write_OD 0x8c5000 thru 0x8c5fff (0x1000 bytes) 000371[0] 1781281342.189472 write_OD 0x7d0000 thru 0x7d0fff (0x1000 bytes) 000372[0] 1781281342.239110 read_OD 0x49a000 thru 0x49efff (0x5000 bytes) 000374[1] 1781281342.253363 mapread 0x01d000 thru 0x0239f3 (0x69f4 bytes) 000375[1] 1781281342.286228 mapread 0x71e000 thru 0x72b1a8 (0xd1a9 bytes) 000376[0] 1781281342.354519 trunc from 0x9e0be9 to 0x3cb7fc (0x6153ec bytes) 000377[0] 1781281342.390875 mapread 0x19f000 thru 0x19f94b (0x94c bytes) 000380[1] 1781281342.470378 trunc from 0x3cb7fd to 0x72b0eb (0x35f8ef bytes) 000382[1] 1781281342.529072 write_OD 0x350000 thru 0x358fff (0x9000 bytes) 000384[1] 1781281342.589069 mapread 0x681000 thru 0x68a858 (0x9859 bytes) 000390[1] 1781281342.638450 write_OD 0x2c9000 thru 0x2d0fff (0x8000 bytes) 000393[1] 1781281342.726326 trunc from 0x72b0ec to 0x48a645 (0x2a0aa6 bytes) 000395[0] 1781281342.806297 write_OD 0x1c3000 thru 0x1cafff (0x8000 bytes) 000399[1] 1781281342.897546 read_OD 0x098000 thru 0x099fff (0x2000 bytes) 000404[1] 1781281342.908182 write_OD 0x42d000 thru 0x42efff (0x2000 bytes) 000406[1] 1781281342.968865 write_OD 0x45d000 thru 0x45efff (0x2000 bytes) 000407[1] 1781281343.037547 mapwrite 0x9ea000 thru 0x9f3417 (0x9418 bytes) 000409[1] 1781281343.453275 write_OD 0x8ba000 thru 0x8bdfff (0x4000 bytes) 000411[0] 1781281343.565061 trunc from 0x9f3418 to 0x5001c5 (0x4f3252 bytes) 000412[1] 1781281343.625866 mapread 0x109000 thru 0x10f271 (0x6272 bytes) 000414[1] 1781281343.700401 write_OD 0x753000 thru 0x755fff (0x3000 bytes) 000416[1] 1781281343.778512 mapread 0x29d000 thru 0x29e1af (0x11b0 bytes) 000417[0] 1781281343.791942 mapread 0x53d000 thru 0x53d3e3 (0x3e4 bytes) 000419[1] 1781281343.891534 trunc from 0x756000 to 0x797a1f (0x41a20 bytes) 000421[1] 1781281343.952862 mapwrite 0x4f9000 thru 0x5015db (0x85dc bytes) 000423[1] 1781281344.116747 mapwrite 0x889000 thru 0x88bd49 (0x2d4a bytes) 000426[1] 1781281344.202208 trunc from 0x88bd4a to 0x06ca43 (0x81f306 bytes) 000428[1] 1781281344.228867 mapwrite 0x276000 thru 0x2853ba (0xf3bb bytes) 000429[1] 1781281344.359177 mapwrite 0x50a000 thru 0x513da5 (0x9da6 bytes) 000430[1] 1781281344.476980 trunc from 0x513da6 to 0x122c72 (0x3f1133 bytes) 000431[1] 1781281344.494231 write_OD 0x192000 thru 0x19cfff (0xb000 bytes) 000433[1] 1781281344.550419 read_OD 0x190000 thru 0x195fff (0x6000 bytes) 000437[0] 1781281344.559190 write_OD 0x900000 thru 0x901fff (0x2000 bytes) 000438[1] 1781281344.610791 mapread 0x4bc000 thru 0x4c3433 (0x7434 bytes) 000439[0] 1781281344.778334 write_OD 0x61c000 thru 0x61cfff (0x1000 bytes) 000440[0] 1781281344.838280 trunc from 0x902000 to 0x997485 (0x95486 bytes) 000441[1] 1781281344.877254 read_OD 0x70c000 thru 0x70dfff (0x2000 bytes) 000442[1] 1781281344.898526 read_OD 0x52f000 thru 0x538fff (0xa000 bytes) 000443[1] 1781281344.927978 mapread 0x322000 thru 0x32b8b1 (0x98b2 bytes) 000444[0] 1781281345.072407 mapread 0x221000 thru 0x22a9d8 (0x99d9 bytes) 000445[1] 1781281345.175590 mapread 0x069000 thru 0x06f594 (0x6595 bytes) 000448[1] 1781281345.235447 read_OD 0x5c1000 thru 0x5cbfff (0xb000 bytes) 000449[1] 1781281345.256471 mapwrite 0x763000 thru 0x7729f6 (0xf9f7 bytes) 000450[1] 1781281345.439076 mapwrite 0x0ec000 thru 0x0f1f91 (0x5f92 bytes) 000454[1] 1781281345.557176 trunc from 0x997486 to 0x08bae1 (0x90b9a4 bytes) 000457[0] 1781281345.584468 write_OD 0x122000 thru 0x124fff (0x3000 bytes) 000459[0] 1781281345.662637 mapwrite 0x2b5000 thru 0x2c0c5d (0xbc5e bytes) 000460[0] 1781281345.933063 write_OD 0x92a000 thru 0x92afff (0x1000 bytes) 000463[1] 1781281346.027278 write_OD 0x9c3000 thru 0x9ccfff (0xa000 bytes) 000467[0] 1781281346.093041 read_OD 0x91b000 thru 0x929fff (0xf000 bytes) 000469[1] 1781281346.102189 read_OD 0x95f000 thru 0x960fff (0x2000 bytes) 000470[0] 1781281346.107984 mapwrite 0x983000 thru 0x985e33 (0x2e34 bytes) 000471[0] 1781281346.219218 mapwrite 0x5b6000 thru 0x5c5a41 (0xfa42 bytes) 000473[0] 1781281346.576450 mapread 0x472000 thru 0x47d943 (0xb944 bytes) 000475[1] 1781281346.706956 trunc from 0x9cd000 to 0x78a042 (0x242fbd bytes) 000476[1] 1781281346.742458 write_OD 0x916000 thru 0x91bfff (0x6000 bytes) 000477[1] 1781281346.799613 mapread 0x877000 thru 0x87e1a5 (0x71a6 bytes) 000479[1] 1781281346.850065 read_OD 0x497000 thru 0x49efff (0x8000 bytes) 000480[1] 1781281346.858051 trunc from 0x91c000 to 0x132925 (0x7e96da bytes) 000485[0] 1781281346.930002 trunc from 0x132926 to 0x134790 (0x1e6b bytes) 000487[1] 1781281346.981750 read_OD 0x043000 thru 0x049fff (0x7000 bytes) 000488[1] 1781281347.009363 mapread 0x076000 thru 0x0824d3 (0xc4d4 bytes) 000490[1] 1781281347.043068 write_OD 0x805000 thru 0x80ffff (0xb000 bytes) 000491[1] 1781281347.133438 mapwrite 0x64f000 thru 0x65479a (0x579b bytes) 000492[0] 1781281347.286916 mapwrite 0x019000 thru 0x026b09 (0xdb0a bytes) 000495[0] 1781281347.506943 write_OD 0x59b000 thru 0x5a1fff (0x7000 bytes) 000496[1] 1781281347.599449 mapread 0x730000 thru 0x73e705 (0xe706 bytes) 000497[1] 1781281347.704120 write_OD 0x800000 thru 0x806fff (0x7000 bytes) 000498[0] 1781281347.757406 write_OD 0x592000 thru 0x59cfff (0xb000 bytes) 000499[1] 1781281347.823208 mapwrite 0x168000 thru 0x16dcae (0x5caf bytes) 000500[0] 1781281348.036145 mapread 0x35a000 thru 0x35e9ee (0x49ef bytes) All operations completed A-OK! PASS 16a (65s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 12:22:33 (1781281353) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.357114 s, 5.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.317422 s, 6.6 MB/s Chance of close/open is 1 in 50 Seed set to 1356 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read skipping zero size read truncating to largest ever: 0x445917 000006[0] 1781281357.437317 trunc from 00000000 to 0x445916 (0x445917 bytes) 000007[1] 1781281357.479241 read 0x161229 thru 0x1624e4 (0x12bc bytes) 000008[0] 1781281357.521218 mapread 0x03a669 thru 0x03ae31 (0x7c9 bytes) 000010[1] 1781281357.545212 trunc from 0x445917 to 0x208b18 (0x23cdfe bytes) 000011[1] 1781281357.637096 write 0x3833e9 thru 0x383595 (0x1ad bytes) 000014[1] 1781281357.695344 read 0x2e5ff0 thru 0x2e7505 (0x1516 bytes) 000017[1] 1781281357.791687 mapread 0x096008 thru 0x097215 (0x120e bytes) 000020[0] 1781281357.854354 mapread 0x13dfe0 thru 0x13f5f7 (0x1618 bytes) 000022[0] 1781281357.931277 mapwrite 0x01bafe thru 0x01d0bd (0x15c0 bytes) truncating to largest ever: 0x6d1c8f 000023[1] 1781281358.088340 trunc from 0x383596 to 0x6d1c8e (0x34e6f9 bytes) 000024[0] 1781281358.137903 mapread 0x40d6eb thru 0x40db91 (0x4a7 bytes) 000025[1] 1781281358.188790 read 0x3ace14 thru 0x3ae93f (0x1b2c bytes) 000028[1] 1781281358.238498 mapread 0x4b1083 thru 0x4b1f9a (0xf18 bytes) 000029[1] 1781281358.296600 write 0x97bf39 thru 0x97cfa9 (0x1071 bytes) 000031[0] 1781281358.340010 mapwrite 0x2d838b thru 0x2d8fa6 (0xc1c bytes) 000032[0] 1781281358.451134 mapread 0x0d4bcb thru 0x0d5d4e (0x1184 bytes) 000033[1] 1781281358.487094 trunc from 0x97cfaa to 0x3c807f (0x5b4f2a bytes) 000034[1] 1781281358.557864 read 0x035697 thru 0x0369cc (0x1336 bytes) 000035[0] 1781281358.652361 read 0x02f2df thru 0x02fe25 (0xb47 bytes) 000036[1] 1781281358.689097 trunc from 0x3c8080 to 0x0b7119 (0x310f66 bytes) 000037[1] 1781281358.778397 read 0x087a6c thru 0x0889f7 (0xf8c bytes) 000039[0] 1781281358.804700 write 0x164a23 thru 0x164d1b (0x2f9 bytes) 000041[1] 1781281358.853981 mapwrite 0x3d49ed thru 0x3d57ad (0xdc1 bytes) 000044[0] 1781281359.057293 read 0x0c8b41 thru 0x0ca969 (0x1e29 bytes) 000046[0] 1781281359.147139 read 0x079166 thru 0x07a6e8 (0x1583 bytes) 000047[0] 1781281359.224195 read 0x383ff0 thru 0x384bcf (0xbe0 bytes) 000052[1] 1781281359.307346 write 0x720a9e thru 0x72140e (0x971 bytes) 000053[0] 1781281359.350005 mapwrite 0x5eba1f thru 0x5ec2ba (0x89c bytes) 000059[0] 1781281359.567921 mapread 0x110d47 thru 0x11153a (0x7f4 bytes) 000062[1] 1781281359.638422 mapread 0x613c1a thru 0x6152c6 (0x16ad bytes) 000063[0] 1781281359.703988 trunc from 0x72140f to 0x63cbc3 (0xe484b bytes) 000064[0] 1781281359.828861 mapwrite 0x0d42eb thru 0x0d5c51 (0x1967 bytes) 000066[0] 1781281359.882942 read 0x0291f3 thru 0x02a63b (0x1449 bytes) 000071[1] 1781281360.053210 read 0x33be53 thru 0x33dd3f (0x1eed bytes) 000073[0] 1781281360.121308 write 0x96588e thru 0x966e54 (0x15c7 bytes) 000074[0] 1781281360.156050 read 0x8afd77 thru 0x8afef1 (0x17b bytes) 000080[0] 1781281360.239884 mapread 0x1a5334 thru 0x1a6792 (0x145f bytes) 000081[1] 1781281360.268046 mapwrite 0x524fb2 thru 0x526264 (0x12b3 bytes) truncating to largest ever: 0x752c86 000082[0] 1781281360.361077 trunc from 0x966e55 to 0x752c85 (0x2141cf bytes) 000083[1] 1781281360.394363 read 0x15903c thru 0x159284 (0x249 bytes) 000085[1] 1781281360.463800 mapwrite 0x9dbef2 thru 0x9dd069 (0x1178 bytes) 000086[1] 1781281360.800810 mapread 0x998da9 thru 0x99916b (0x3c3 bytes) 000089[0] 1781281360.853738 mapwrite 0x2cda7d thru 0x2cf7e9 (0x1d6d bytes) 000094[1] 1781281361.076016 trunc from 0x9dd06a to 0x4361f2 (0x5a6e77 bytes) 000095[1] 1781281361.173532 mapread 0x3a5ac1 thru 0x3a66bf (0xbff bytes) 000096[1] 1781281361.181018 mapread 0x0fdb30 thru 0x0feb17 (0xfe8 bytes) 000102[1] 1781281361.296344 mapwrite 0x62a1d3 thru 0x62b470 (0x129e bytes) 000103[1] 1781281361.367955 mapwrite 0x7ad6b8 thru 0x7aefb6 (0x18ff bytes) 000109[1] 1781281361.537972 write 0x3509b6 thru 0x350ee5 (0x530 bytes) 000113[1] 1781281361.539754 mapread 0x5b515c thru 0x5b5d22 (0xbc7 bytes) 000114[1] 1781281361.546342 write 0x917aea thru 0x918f95 (0x14ac bytes) 000115[1] 1781281361.580169 mapwrite 0x34d8d3 thru 0x34dc9a (0x3c8 bytes) 000116[0] 1781281361.629577 mapwrite 0x049a2d thru 0x04aa0f (0xfe3 bytes) 000122[0] 1781281361.787831 read 0x632e31 thru 0x634b2b (0x1cfb bytes) truncating to largest ever: 0x83d667 000123[0] 1781281361.826019 trunc from 0x918f96 to 0x83d666 (0xdb92f bytes) 000125[0] 1781281361.919777 mapread 0x597053 thru 0x598259 (0x1207 bytes) 000127[1] 1781281361.993161 write 0x1503b6 thru 0x151bc5 (0x1810 bytes) 000129[1] 1781281362.049832 trunc from 0x83d667 to 0x7dc439 (0x6122d bytes) 000135[0] 1781281362.201473 mapwrite 0x7aba5f thru 0x7ad463 (0x1a05 bytes) 000139[0] 1781281362.376199 mapwrite 0x0f094d thru 0x0f195a (0x100e bytes) 000140[1] 1781281362.475471 mapwrite 0x02390d thru 0x024f91 (0x1685 bytes) 000141[1] 1781281362.582385 read 0x32ef5e thru 0x3308dc (0x197f bytes) 000142[0] 1781281362.711385 write 0x385f3f thru 0x38647b (0x53d bytes) 000143[0] 1781281362.766895 write 0x126d15 thru 0x12714c (0x438 bytes) 000145[0] 1781281362.863749 read 0x7b85b9 thru 0x7b9a18 (0x1460 bytes) 000146[1] 1781281362.907262 trunc from 0x7dc43a to 0x5eed89 (0x1ed6b0 bytes) 000147[0] 1781281363.017693 mapwrite 0x9780c5 thru 0x9798cc (0x1808 bytes) 000149[1] 1781281363.240035 mapwrite 0x89efdc thru 0x8a0827 (0x184c bytes) 000150[1] 1781281363.385297 trunc from 0x9798cd to 0x35eac4 (0x61ae08 bytes) 000151[0] 1781281363.429794 read 0x1f6fea thru 0x1f8333 (0x134a bytes) 000152[1] 1781281363.463287 mapwrite 0x4a2c28 thru 0x4a3110 (0x4e9 bytes) 000153[1] 1781281363.568979 write 0x4a1add thru 0x4a3999 (0x1ebd bytes) 000155[0] 1781281363.622874 mapread 0x474812 thru 0x4749bc (0x1ab bytes) 000159[0] 1781281363.794667 mapwrite 0x208966 thru 0x208b99 (0x234 bytes) 000161[1] 1781281364.018220 read 0x03e89f thru 0x03ff61 (0x16c3 bytes) 000162[1] 1781281364.164215 trunc from 0x4a399a to 0x469415 (0x3a584 bytes) 000165[1] 1781281364.269830 mapwrite 0x3a8400 thru 0x3a9c20 (0x1821 bytes) 000166[1] 1781281364.327837 read 0x284ed5 thru 0x2856f6 (0x822 bytes) 000169[0] 1781281364.391744 trunc from 0x469416 to 0x1a4686 (0x2c4d8f bytes) 000170[1] 1781281364.446226 mapread 0x08ea4f thru 0x08f1b6 (0x768 bytes) 000172[1] 1781281364.485394 trunc from 0x1a4687 to 0x59f64f (0x3fafc9 bytes) truncating to largest ever: 0x9c1402 000173[1] 1781281364.532793 trunc from 0x59f650 to 0x9c1401 (0x421db2 bytes) 000176[0] 1781281364.634820 mapread 0x35a7df thru 0x35a906 (0x128 bytes) 000182[1] 1781281364.788921 mapwrite 0x81ede9 thru 0x81f03a (0x252 bytes) 000183[0] 1781281365.039969 mapwrite 0x60b328 thru 0x60b348 (0x21 bytes) 000184[0] 1781281365.171155 mapwrite 0x259140 thru 0x25a066 (0xf27 bytes) truncating to largest ever: 0x9c8116 000185[1] 1781281365.286384 trunc from 0x9c1402 to 0x9c8115 (0x6d14 bytes) 000186[0] 1781281365.357793 mapwrite 0x9c0550 thru 0x9c062b (0xdc bytes) 000187[1] 1781281365.485889 write 0x95898c thru 0x95a5cc (0x1c41 bytes) 000188[1] 1781281365.519255 trunc from 0x9c8116 to 0x603b80 (0x3c4595 bytes) 000189[1] 1781281365.609881 mapread 0x309f9b thru 0x30a288 (0x2ee bytes) 000192[0] 1781281365.695609 read 0x540166 thru 0x5418d0 (0x176b bytes) 000195[1] 1781281365.809794 read 0x4222fa thru 0x422fb2 (0xcb9 bytes) 000196[1] 1781281365.876827 read 0x5b15f2 thru 0x5b1c28 (0x637 bytes) 000197[1] 1781281365.928070 trunc from 0x603b81 to 0x377f33 (0x28bc4d bytes) 000200[0] 1781281366.108849 read 0x15024b thru 0x152165 (0x1f1b bytes) 000203[1] 1781281366.283042 mapwrite 0x93e1d7 thru 0x93f316 (0x1140 bytes) 000204[1] 1781281366.444673 mapwrite 0x3404f1 thru 0x34230d (0x1e1d bytes) 000205[0] 1781281366.527518 mapwrite 0x8d411d thru 0x8d5a27 (0x190b bytes) 000210[1] 1781281366.746507 mapread 0x377a21 thru 0x37944c (0x1a2c bytes) 000212[0] 1781281366.816230 read 0x5f6b0d thru 0x5f6e9b (0x38f bytes) 000215[1] 1781281367.042243 read 0x16955a thru 0x16a3e0 (0xe87 bytes) 000217[1] 1781281367.067535 mapread 0x52ee3a thru 0x530993 (0x1b5a bytes) 000218[1] 1781281367.106847 trunc from 0x93f317 to 0x652ad2 (0x2ec844 bytes) 000219[1] 1781281367.172984 write 0x09441e thru 0x0953fb (0xfde bytes) 000221[1] 1781281367.294775 mapread 0x420b52 thru 0x42262b (0x1ada bytes) 000222[0] 1781281367.319491 read 0x061eec thru 0x0635b9 (0x16ce bytes) 000224[0] 1781281367.369655 trunc from 0x652ad3 to 0x436871 (0x21c261 bytes) 000226[0] 1781281367.439501 mapwrite 0x5f1400 thru 0x5f1a5f (0x660 bytes) 000227[0] 1781281367.644025 mapwrite 0x121163 thru 0x1224a8 (0x1346 bytes) 000228[0] 1781281367.710690 write 0x66e0b2 thru 0x66f5b3 (0x1502 bytes) 000229[1] 1781281367.717018 mapwrite 0x169a44 thru 0x169b82 (0x13f bytes) 000233[0] 1781281367.864158 read 0x46d083 thru 0x46e592 (0x1510 bytes) 000234[1] 1781281367.884340 trunc from 0x66f5b4 to 0x4b5c0d (0x1b99a6 bytes) 000236[0] 1781281367.962067 trunc from 0x4b5c0e to 0x8b59ff (0x3ffdf2 bytes) 000238[1] 1781281368.050583 read 0x634472 thru 0x635726 (0x12b5 bytes) 000239[0] 1781281368.115760 trunc from 0x8b5a00 to 0x17ea3f (0x736fc0 bytes) 000240[1] 1781281368.172661 mapwrite 0x9b5f18 thru 0x9b6195 (0x27e bytes) 000241[1] 1781281368.421207 trunc from 0x9b6196 to 0x8ad7cf (0x1089c6 bytes) 000242[0] 1781281368.476433 read 0x5a0535 thru 0x5a1d08 (0x17d4 bytes) 000243[0] 1781281368.526796 mapread 0x3a884e thru 0x3a911a (0x8cd bytes) 000244[1] 1781281368.553058 trunc from 0x8ad7d0 to 0x3ed29e (0x4c0531 bytes) 000245[0] 1781281368.604015 write 0x176199 thru 0x177688 (0x14f0 bytes) 000246[1] 1781281368.643324 mapwrite 0x72d41f thru 0x72f16a (0x1d4c bytes) 000247[1] 1781281368.783040 trunc from 0x72f16b to 0x7f80b9 (0xc8f4f bytes) 000248[1] 1781281368.849495 trunc from 0x7f80ba to 0x9514c7 (0x15940e bytes) 000249[1] 1781281368.888753 read 0x418f85 thru 0x4190e4 (0x160 bytes) 000250[0] 1781281369.078296 write 0x42887e thru 0x42a7ed (0x1f70 bytes) 000251[0] 1781281369.198753 trunc from 0x9514c8 to 0x2440ba (0x70d40d bytes) 000252[1] 1781281369.260474 mapwrite 0x6be939 thru 0x6c05b5 (0x1c7d bytes) 000253[0] 1781281369.430446 mapread 0x30df66 thru 0x30fe9a (0x1f35 bytes) 000254[0] 1781281369.550014 mapwrite 0x3a0133 thru 0x3a161e (0x14ec bytes) 000255[0] 1781281369.701115 mapread 0x4aa182 thru 0x4ab616 (0x1495 bytes) 000256[0] 1781281369.730935 mapread 0x6ac4a2 thru 0x6ad28d (0xdec bytes) 000257[0] 1781281369.774812 trunc from 0x6c05b6 to 0x70c360 (0x4bdab bytes) 000262[0] 1781281369.858111 write 0x75c376 thru 0x75c49d (0x128 bytes) 000265[1] 1781281369.886263 mapwrite 0x0b8d5b thru 0x0b979f (0xa45 bytes) 000266[0] 1781281369.982831 read 0x00538a thru 0x00687d (0x14f4 bytes) 000268[0] 1781281370.172451 mapwrite 0x38ee6a thru 0x38f220 (0x3b7 bytes) 000269[1] 1781281370.259040 read 0x5ae76b thru 0x5ae949 (0x1df bytes) 000272[1] 1781281370.321587 read 0x40589a thru 0x406bcc (0x1333 bytes) 000275[0] 1781281370.388254 mapwrite 0x5b2f2e thru 0x5b4ea6 (0x1f79 bytes) 000276[1] 1781281370.478061 mapread 0x089353 thru 0x08a772 (0x1420 bytes) 000280[0] 1781281370.545443 mapread 0x1f8968 thru 0x1f9cd3 (0x136c bytes) 000281[1] 1781281370.557486 mapwrite 0x1ef842 thru 0x1f1338 (0x1af7 bytes) 000282[0] 1781281370.644691 read 0x1e1a6d thru 0x1e32b9 (0x184d bytes) 000284[1] 1781281370.722174 mapread 0x3a5d66 thru 0x3a62d0 (0x56b bytes) 000285[0] 1781281370.731626 read 0x0d88cb thru 0x0da149 (0x187f bytes) 000288[0] 1781281370.734867 trunc from 0x75c49e to 0x52f453 (0x22d04a bytes) 000289[0] 1781281370.861152 read 0x359552 thru 0x359d8b (0x83a bytes) 000294[1] 1781281371.150209 mapwrite 0x14e363 thru 0x14ff5c (0x1bfa bytes) 000295[1] 1781281371.289957 mapread 0x4a7737 thru 0x4a795b (0x225 bytes) 000297[0] 1781281371.354481 read 0x24c412 thru 0x24d3f2 (0xfe1 bytes) 000298[0] 1781281371.445064 read 0x51da97 thru 0x51df8b (0x4f5 bytes) 000300[1] 1781281371.510092 read 0x2816eb thru 0x282227 (0xb3d bytes) 000304[1] 1781281371.621044 mapread 0x33e68d thru 0x33fc70 (0x15e4 bytes) 000305[0] 1781281371.621737 mapread 0x08b21c thru 0x08b74c (0x531 bytes) 000306[0] 1781281371.629666 trunc from 0x52f454 to 0x13a751 (0x3f4d02 bytes) 000307[1] 1781281371.700267 write 0x77ee58 thru 0x77fc64 (0xe0d bytes) 000308[0] 1781281371.732689 write 0x59bd6a thru 0x59c58f (0x826 bytes) 000315[1] 1781281371.877529 trunc from 0x77fc65 to 0x3d6fa6 (0x3a8cbe bytes) 000316[0] 1781281371.932035 mapread 0x2d140a thru 0x2d2abb (0x16b2 bytes) 000317[0] 1781281371.983387 mapread 0x2a90b0 thru 0x2aa41b (0x136c bytes) 000319[1] 1781281372.024062 mapwrite 0x7fe84e thru 0x8001ce (0x1981 bytes) 000320[0] 1781281372.167733 mapwrite 0x46bc12 thru 0x46c437 (0x826 bytes) 000321[0] 1781281372.265748 mapread 0x2016a4 thru 0x201aca (0x427 bytes) 000323[0] 1781281372.309463 trunc from 0x8001cf to 0x079548 (0x786c86 bytes) 000324[1] 1781281372.348240 mapread 0x0014c9 thru 0x002132 (0xc6a bytes) 000325[1] 1781281372.481217 read 0x0234b1 thru 0x0253b8 (0x1f08 bytes) 000330[1] 1781281372.596715 mapwrite 0x01ae1c thru 0x01ba77 (0xc5c bytes) 000331[1] 1781281372.767422 read 0x009e38 thru 0x00b3d8 (0x15a1 bytes) 000333[1] 1781281372.839417 trunc from 0x079549 to 0x8e84ee (0x86efa6 bytes) 000334[1] 1781281372.917027 mapread 0x6e4a0d thru 0x6e4e00 (0x3f4 bytes) 000335[0] 1781281373.124779 mapread 0x6cc4a0 thru 0x6ccff3 (0xb54 bytes) 000336[0] 1781281373.237728 write 0x9d82ca thru 0x9d89ea (0x721 bytes) 000340[0] 1781281373.344770 trunc from 0x9d89eb to 0x63ccb9 (0x39bd31 bytes) 000341[1] 1781281373.401923 write 0x06782a thru 0x067884 (0x5b bytes) 000342[0] 1781281373.504139 write 0x8ef126 thru 0x8f107f (0x1f5a bytes) 000343[0] 1781281373.570843 write 0x01b292 thru 0x01cd6a (0x1ad9 bytes) 000347[1] 1781281373.767048 write 0x88c9c9 thru 0x88d8d6 (0xf0e bytes) 000349[0] 1781281373.843955 read 0x7155b5 thru 0x7167b5 (0x1201 bytes) 000350[0] 1781281373.889219 mapread 0x69a2e8 thru 0x69b97a (0x1693 bytes) 000353[0] 1781281374.022229 write 0x5b21c5 thru 0x5b33ea (0x1226 bytes) 000355[0] 1781281374.072822 write 0x702be7 thru 0x7041ab (0x15c5 bytes) 000356[0] 1781281374.108403 read 0x770276 thru 0x770e7c (0xc07 bytes) 000358[1] 1781281374.145463 mapwrite 0x1e65cd thru 0x1e8536 (0x1f6a bytes) 000359[0] 1781281374.286982 mapread 0x843c4c thru 0x844df9 (0x11ae bytes) 000360[0] 1781281374.295800 mapwrite 0x2c45a7 thru 0x2c49b6 (0x410 bytes) 000363[1] 1781281374.485155 trunc from 0x8f1080 to 0x0c7496 (0x829be9 bytes) 000364[1] 1781281374.530544 trunc from 0x0c7497 to 0x53ed1e (0x477888 bytes) 000367[1] 1781281374.595781 read 0x0be386 thru 0x0bee2f (0xaaa bytes) 000368[0] 1781281374.726546 mapread 0x0ed5ad thru 0x0ef19c (0x1bf0 bytes) 000372[1] 1781281374.822320 mapread 0x25a4db thru 0x25c1e7 (0x1d0d bytes) 000373[1] 1781281374.860380 read 0x19854a thru 0x19a0af (0x1b66 bytes) 000378[0] 1781281374.938941 trunc from 0x53ed1f to 0x55672c (0x17a0e bytes) 000380[0] 1781281375.024044 read 0x3eb2d9 thru 0x3ec426 (0x114e bytes) 000381[1] 1781281375.069508 mapread 0x53cfb7 thru 0x53de03 (0xe4d bytes) 000382[1] 1781281375.119274 mapread 0x05f5dc thru 0x060129 (0xb4e bytes) 000383[1] 1781281375.158690 mapread 0x3cd316 thru 0x3cec71 (0x195c bytes) 000384[1] 1781281375.167276 mapwrite 0x04d3ec thru 0x04ec6e (0x1883 bytes) 000385[0] 1781281375.234936 mapwrite 0x3b0b7f thru 0x3b2823 (0x1ca5 bytes) 000386[1] 1781281375.356545 read 0x22216b thru 0x2231af (0x1045 bytes) 000388[0] 1781281375.456066 mapwrite 0x0a445a thru 0x0a48bf (0x466 bytes) 000389[0] 1781281375.608252 trunc from 0x55672d to 0x370478 (0x1e62b4 bytes) 000390[1] 1781281375.671234 write 0x6a50ec thru 0x6a6228 (0x113d bytes) 000392[0] 1781281375.767369 mapread 0x68895f thru 0x689893 (0xf35 bytes) 000393[0] 1781281375.875857 mapread 0x5753a3 thru 0x577298 (0x1ef6 bytes) 000395[0] 1781281375.909165 read 0x186246 thru 0x1876a1 (0x145c bytes) 000398[0] 1781281376.103944 mapwrite 0x69966f thru 0x69b4ba (0x1e4c bytes) 000400[0] 1781281376.303215 trunc from 0x6a6229 to 0x13e803 (0x567a25 bytes) 000404[0] 1781281376.431374 mapread 0x047588 thru 0x048766 (0x11df bytes) 000410[0] 1781281376.625085 mapread 0x0924cd thru 0x092f2e (0xa62 bytes) 000411[1] 1781281376.653307 trunc from 0x13e804 to 0x4b22d5 (0x373ad2 bytes) 000412[0] 1781281376.748429 write 0x4deef2 thru 0x4e09ac (0x1abb bytes) 000413[0] 1781281376.779761 mapread 0x27f579 thru 0x281473 (0x1efb bytes) 000415[1] 1781281376.884202 trunc from 0x4e09ad to 0x3ab811 (0x13519b bytes) 000416[1] 1781281376.960931 read 0x2ff1ab thru 0x2ff882 (0x6d8 bytes) 000422[0] 1781281377.058386 mapread 0x24f13d thru 0x24f442 (0x306 bytes) 000423[1] 1781281377.089201 mapread 0x0e6404 thru 0x0e76da (0x12d7 bytes) 000424[0] 1781281377.097732 trunc from 0x3ab812 to 0x733205 (0x3879f4 bytes) 000425[1] 1781281377.147939 trunc from 0x733206 to 0x59e0eb (0x19511a bytes) 000427[0] 1781281377.233924 write 0x83dac1 thru 0x83e066 (0x5a6 bytes) 000428[0] 1781281377.258654 mapwrite 0x2b3145 thru 0x2b45a7 (0x1463 bytes) 000429[0] 1781281377.418764 trunc from 0x83e067 to 0x66a7be (0x1d38a8 bytes) 000432[0] 1781281377.552883 write 0x699be5 thru 0x69a96b (0xd87 bytes) 000434[1] 1781281377.608596 mapread 0x5269fa thru 0x527b0c (0x1113 bytes) 000435[1] 1781281377.670962 mapread 0x071403 thru 0x0726eb (0x12e9 bytes) 000436[0] 1781281377.769238 read 0x0adc60 thru 0x0ae3db (0x77c bytes) 000438[0] 1781281377.790283 mapwrite 0x54cf38 thru 0x54d505 (0x5ce bytes) 000439[0] 1781281377.994057 write 0x214cbf thru 0x215ee1 (0x1223 bytes) 000441[1] 1781281378.059918 mapread 0x0cafe2 thru 0x0cbce7 (0xd06 bytes) 000442[0] 1781281378.113000 trunc from 0x69a96c to 0x7da099 (0x13f72e bytes) 000443[1] 1781281378.192207 trunc from 0x7da09a to 0x23bbdf (0x59e4ba bytes) 000444[0] 1781281378.250405 mapwrite 0x29cf2a thru 0x29dfd0 (0x10a7 bytes) 000445[0] 1781281378.377075 write 0x46ec28 thru 0x470692 (0x1a6b bytes) 000447[0] 1781281378.384200 trunc from 0x470693 to 0x334392 (0x13c300 bytes) 000453[1] 1781281378.495216 trunc from 0x334393 to 0x353514 (0x1f182 bytes) 000454[1] 1781281378.576875 trunc from 0x353515 to 0x058aa3 (0x2faa71 bytes) 000455[1] 1781281378.638053 write 0x2aadd1 thru 0x2ac6ff (0x192f bytes) 000456[0] 1781281378.691246 mapread 0x236adc thru 0x237bba (0x10df bytes) 000457[1] 1781281378.808495 mapread 0x01059d thru 0x01141c (0xe80 bytes) 000459[1] 1781281378.893073 read 0x1011cd thru 0x101dad (0xbe1 bytes) 000460[1] 1781281378.954660 trunc from 0x2ac700 to 0x8ce90e (0x62220f bytes) 000464[1] 1781281379.054444 write 0x60a8be thru 0x60b747 (0xe8a bytes) 000467[0] 1781281379.253044 mapwrite 0x82299a thru 0x824084 (0x16eb bytes) 000468[1] 1781281379.440397 read 0x874a4e thru 0x875a74 (0x1027 bytes) 000469[1] 1781281379.506087 mapread 0x57b4f2 thru 0x57d155 (0x1c64 bytes) 000471[1] 1781281379.539232 mapwrite 0x00df71 thru 0x00e9fb (0xa8b bytes) 000472[0] 1781281379.659106 read 0x343e0d thru 0x344733 (0x927 bytes) 000473[1] 1781281379.750691 mapread 0x6e381e thru 0x6e3a61 (0x244 bytes) 000475[0] 1781281379.830805 read 0x89c51f thru 0x89e1a3 (0x1c85 bytes) 000476[1] 1781281379.892034 write 0x08728f thru 0x0875fb (0x36d bytes) 000478[1] 1781281379.947373 read 0x5accb0 thru 0x5aeb5c (0x1ead bytes) 000481[1] 1781281380.063715 trunc from 0x8ce90f to 0x18ae3b (0x743ad3 bytes) 000482[1] 1781281380.126948 write 0x78913d thru 0x789a05 (0x8c9 bytes) 000483[0] 1781281380.133732 mapread 0x1cdb88 thru 0x1cecaf (0x1128 bytes) 000484[1] 1781281380.206516 mapread 0x436368 thru 0x43669a (0x333 bytes) 000485[1] 1781281380.242148 mapread 0x73baf9 thru 0x73da19 (0x1f21 bytes) 000487[1] 1781281380.321230 mapread 0x699ffc thru 0x69bdb8 (0x1dbd bytes) 000490[0] 1781281380.366221 mapwrite 0x79c702 thru 0x79daf9 (0x13f8 bytes) 000493[0] 1781281380.497366 trunc from 0x79dafa to 0x17a7c4 (0x623335 bytes) 000495[1] 1781281380.560121 mapwrite 0x10b2f0 thru 0x10bc22 (0x933 bytes) 000496[0] 1781281380.778873 write 0x1b0279 thru 0x1b0766 (0x4ee bytes) All operations completed A-OK! PASS 16b (33s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 12:23:06 (1781281386) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (5s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 12:23:11 (1781281391) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.428648 s, 4.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.308473 s, 6.8 MB/s ldlm.namespaces.MGC192.168.203.114@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8dfe86913000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8dfe90fcc800.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff8dfe86913000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff8dfe90fcc800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8dfe90fcc800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8dfe90fcc800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 14.5509 s, 7.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.0848 s, 25.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.50415 s, 29.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.32967 s, 14.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.67461 s, 10.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.7379 s, 28.1 MB/s PASS 16d (57s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 12:24:08 (1781281448) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.37659 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148175 s, 7.1 MB/s PASS 16e (8s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 12:24:16 (1781281456) TIMEOUT PASS 16f (29s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 12:24:45 (1781281485) TIMEOUT PASS 16g (29s) == sanityn test 16i: read after truncate file ============ 12:25:14 (1781281514) striped dir -i0 -c2 -H all_char /mnt/lustre/d16i.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (8s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 12:25:23 (1781281523) fail_loc=0x8000030a PASS 17 (11s) == sanityn test 18: mmap sanity check =========================================================================================== 12:25:33 (1781281533) mmap test1: basic mmap operation (PASS, 0.17951s) mmap test2: MAP_PRIVATE not write back (PASS, 0.084664s) mmap test3: concurrent mmap ops on two nodes (PASS, 3.3007s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.4376s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.15543s) mmap test6: check mmap write/read content on two nodes (PASS, 0.37817s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.1136s) PASS 18 (16s) == sanityn test 19: test concurrent uncached read races ========================================================================= 12:25:49 (1781281549) oleg314-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg314-client: oleg314-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (5s) == sanityn test 20: test extra readahead page left in cache ============================================================== 12:25:55 (1781281555) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d20.sanityn PASS 20 (8s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 12:26:03 (1781281563) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d21.sanityn rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (7s) == sanityn test 23: others should see updated atime while another read============================================================== 12:26:10 (1781281570) now is 1781281573 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 new atime is 1781281634 PASS 23 (71s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 12:27:21 (1781281641) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 484.1K 386 483.7K 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 484.6K 333 484.3K 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 115.3K 437 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.4K 442 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 484.1K 386 483.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 484.6K 333 484.3K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 115.3K 437 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.4K 442 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.6M 2.1G 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495689 386 495303 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 496261 333 495928 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 118101 437 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118138 442 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 236079 719 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 484.1K 386 483.7K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 484.6K 333 484.3K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 115.3K 437 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.4K 442 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495689 386 495303 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 496261 333 495928 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 118101 437 117664 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118138 442 117696 1% /mnt/lustre[OST:1] filesystem_summary: 236079 719 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495689 386 495303 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 496261 333 495928 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 118101 437 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118138 442 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 236079 719 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (8s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 12:27:29 (1781281649) PASS 24b (7s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 12:27:36 (1781281656) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (9s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 12:27:45 (1781281665) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] PASS 25b (9s) == sanityn test 26a: allow mtime to get older ============ 12:27:54 (1781281674) utime: good mknod times 1781281675 <= 1781281676 <= 1781281676 for /mnt/lustre/f26a utime: good mknod times 1781281675 <= 1781281676 <= 1781281676 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (9s) == sanityn test 26b: sync mtime between ost and mds ====== 12:28:03 (1781281683) PASS 26b (9s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 12:28:16 (1781281696) striped dir -i0 -c2 -H crush /mnt/lustre/d30.sanityn 0 PASS 30 (8s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 12:28:25 (1781281705) striped dir -i1 -c2 -H all_char /mnt/lustre/d31a.sanityn fail_loc=0x314 PASS 31a (8s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 12:28:33 (1781281713) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg314-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete striped dir -i1 -c2 -H crush /mnt/lustre/d31b.sanityn fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.114@tcp:/lustre 7542784 12288 7522304 1% /mnt/lustre2 PASS 31b (24s) == sanityn test 31r: open-rename(replace) race =========== 12:28:57 (1781281737) -rw-r--r-- 1 root root 0 Jun 12 12:28 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (10s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 12:29:09 (1781281749) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (3s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 12:29:12 (1781281752) Stopping /mnt/lustre-mds1 (opts:) on oleg314-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg314-server: oleg314-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 1 Started lustre-MDT0000 sync_count 2 sync_count 0 PASS 33c (41s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 12:29:53 (1781281793) CoS count 2 CoS count 2 CoS count 2 CoS count 2 CoS count 2 CoS count 4 CoS count 3 CoS count 4 PASS 33d (126s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 12:31:59 (1781281919) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (3s) == sanityn test 34: no lock timeout under IO ============= 12:32:02 (1781281922) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8dfe86913000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8dfe90fcc800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8dfe86913000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8dfe90fcc800.ost_server_uuid in FULL state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8dfe86913000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8dfe90fcc800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8dfe86913000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8dfe90fcc800.ost_server_uuid in IDLE state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8dfe86913000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8dfe90fcc800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8dfe86913000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8dfe90fcc800.ost_server_uuid in FULL state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8dfe86913000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8dfe90fcc800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8dfe86913000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8dfe90fcc800.ost_server_uuid in IDLE state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8dfe86913000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8dfe90fcc800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8dfe86913000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8dfe90fcc800.ost_server_uuid in FULL state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8dfe86913000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8dfe90fcc800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8dfe86913000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8dfe90fcc800.ost_server_uuid in IDLE state after 0 sec PASS 34 (126s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 12:34:08 (1781282048) striped dir -i1 -c2 -H crush /mnt/lustre/d35.sanityn fail_loc=0x80000317 Race attempt 0 Wait for 52340 52361 for 60 sec... - open/close 672 (time 1781282065.44 total 10.00 last 67.17) - open/close 1343 (time 1781282075.45 total 20.01 last 67.03) - open/close 2038 (time 1781282085.45 total 30.02 last 69.48) - open/close 2764 (time 1781282095.45 total 40.02 last 72.58) - open/close 3517 (time 1781282105.46 total 50.03 last 75.27) total: 4000 open/close in 56.53 seconds: 70.76 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.114@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.203.114@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (73s) == sanityn test 36: handle ESTALE/open-unlink correctly == 12:35:21 (1781282121) striped dir -i0 -c2 -H crush /mnt/lustre/d36.sanityn Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg314-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.38211 s, 37.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.05091 s, 49.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.13055 s, 46.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.35357 s, 38.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7520256) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.972308 s, 53.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.42452 s, 36.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.13074 s, 46.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.00555 s, 52.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7522304) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.45955 s, 35.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.11966 s, 46.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.10813 s, 47.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7524352) after_dd(7471104) after(7522304) PASS 36 (297s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 12:40:18 (1781282418) striped dir -i1 -c2 -H all_char /mnt/lustre/d37.sanityn multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 - create 1461 (time 1781282432.22 total 10.00 last 146.06) - create 2822 (time 1781282442.22 total 20.00 last 136.09) - create 4342 (time 1781282452.23 total 30.00 last 151.98) - create 5682 (time 1781282462.23 total 40.01 last 133.94) - create 7009 (time 1781282472.23 total 50.01 last 132.65) - create 8379 (time 1781282482.23 total 60.01 last 136.99) - create 9837 (time 1781282492.24 total 70.02 last 145.73) total: 10000 create in 71.13 seconds: 140.58 ops/second PASS 37 (82s) == sanityn test 39a: test from 11063 ============================================================================================ 12:41:40 (1781282500) repeat after cancel_lru_locks PASS 39a (6s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 12:41:46 (1781282506) repeat after cancel_lru_locks PASS 39b (8s) == sanityn test 39c: check truncate mtime update ================================================================================ 12:41:54 (1781282514) repeat after cancel_lru_locks PASS 39c (7s) == sanityn test 39d: sync write should update mtime ====== 12:42:02 (1781282522) fail_loc=0x411 fail_loc=0 PASS 39d (6s) == sanityn test 40a: pdirops: create vs others ======================================================================== 12:42:08 (1781282528) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (16s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 12:42:24 (1781282544) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (19s) == sanityn test 40c: pdirops: link and others ======================================================================== 12:42:43 (1781282563) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (17s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 12:43:01 (1781282581) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (17s) == sanityn test 40e: pdirops: rename and others ======================================================================== 12:43:17 (1781282597) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (18s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 12:43:35 (1781282615) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (14s) == sanityn test 41b: pdirops: create vs create ======================================================================== 12:43:49 (1781282629) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 41b (12s) == sanityn test 41c: pdirops: create vs link ======================================================================== 12:44:01 (1781282641) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (12s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 12:44:13 (1781282653) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41d (11s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 12:44:24 (1781282664) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41e (12s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 12:44:36 (1781282676) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41f (13s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 12:44:49 (1781282689) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41g (13s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 12:45:02 (1781282702) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41h (15s) == sanityn test 41i: reint_open: create vs create ======== 12:45:17 (1781282717) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (1133s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 13:04:10 (1781283850) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (15s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 13:04:26 (1781283866) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 42b (17s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 13:04:43 (1781283883) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (17s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 13:05:01 (1781283901) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42d (18s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 13:05:19 (1781283919) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (17s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 13:05:36 (1781283936) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42f (17s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 13:05:53 (1781283953) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42g (16s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 13:06:10 (1781283970) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42h (18s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 13:06:27 (1781283987) PASS 43a (233s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 13:10:21 (1781284221) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43b (20s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 13:10:40 (1781284240) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43c (18s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 13:10:58 (1781284258) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (18s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 13:11:16 (1781284276) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43e (17s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 13:11:34 (1781284294) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (17s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 13:11:50 (1781284310) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (15s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 13:12:05 (1781284325) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43h (16s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 13:12:21 (1781284341) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43i (15s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 13:12:36 (1781284356) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (166s) == sanityn test 43k: unlink vs create ==================== 13:15:22 (1781284522) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (1420s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 13:39:02 (1781285942) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Jun 12 13:39:11 EDT 2026 PASS 44a (14s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 13:39:16 (1781285956) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 44b (14s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 13:39:30 (1781285970) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (14s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 13:39:44 (1781285984) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44d (14s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 13:39:58 (1781285998) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44e (13s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 13:40:11 (1781286011) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44f (12s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 13:40:23 (1781286023) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44g (13s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 13:40:36 (1781286036) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44h (14s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 13:40:50 (1781286050) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f44i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f44i.sanityn': File exists Conflict PASS 44i (13s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 13:41:03 (1781286063) PASS 45a (213s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 13:44:36 (1781286276) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45b (12s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 13:44:48 (1781286288) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45c (13s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 13:45:01 (1781286301) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (12s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 13:45:14 (1781286314) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45e (13s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 13:45:27 (1781286327) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (13s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 13:45:40 (1781286340) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (13s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 13:45:54 (1781286354) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45h (12s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 13:46:06 (1781286366) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45i (15s) == sanityn test 45j: read vs rename ====================== 13:46:21 (1781286381) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (1467s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 14:10:48 (1781287848) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (14s) == sanityn test 46b: pdirops: link vs create ======================================================================== 14:11:03 (1781287863) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 46b (13s) == sanityn test 46c: pdirops: link vs link ======================================================================== 14:11:16 (1781287876) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (14s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 14:11:30 (1781287890) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46d (14s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 14:11:44 (1781287904) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46e (15s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 14:11:59 (1781287919) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46f (15s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 14:12:14 (1781287934) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46g (14s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 14:12:28 (1781287948) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46h (13s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 14:12:41 (1781287961) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f46i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f46i.sanityn': File exists Conflict PASS 46i (13s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 14:12:55 (1781287975) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f47a.sanityn': File exists Conflict PASS 47a (13s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 14:13:07 (1781287987) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 47b (15s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 14:13:22 (1781288002) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f47c.sanityn' to '/mnt/lustre2/f47c.sanityn-2': File exists Conflict PASS 47c (14s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 14:13:36 (1781288016) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47d (14s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 14:13:50 (1781288030) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f47e.sanityn' with non-directory Conflict PASS 47e (13s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 14:14:03 (1781288043) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47f (14s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 14:14:17 (1781288057) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47g (14s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 14:14:31 (1781288071) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0528575 s, 194 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (11s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 14:14:42 (1781288082) 0+1 records in 0+1 records out 158 bytes copied, 0.0293288 s, 5.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (263545) - No such process PASS 51a (7s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 14:14:49 (1781288089) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0133244 s, 76.9 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785199 s, 13.4 MB/s 3145728 PASS 51b (19s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 14:15:08 (1781288108) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0204745 s, 50.0 kB/s PASS 51c (13s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 14:15:21 (1781288121) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404098 s, 25.9 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (8s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 14:15:29 (1781288129) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1255 0x4e7 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1255 0x4e7 0 read lease(1) has applied. PASS 51e (9s) == sanityn test 54: rename locking ======================= 14:15:39 (1781288139) llite.lustre-ffff8dfe86913000.xattr_cache=1 llite.lustre-ffff8dfe90fcc800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281058 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:15:40.000000000 -0400 Modify: 2026-06-12 14:15:40.000000000 -0400 Change: 2026-06-12 14:15:40.000000000 -0400 Birth: 2026-06-12 14:15:40.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762657 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:15:47.000000000 -0400 Modify: 2026-06-12 14:15:49.000000000 -0400 Change: 2026-06-12 14:15:49.000000000 -0400 Birth: 2026-06-12 14:15:47.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281061 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:15:54.000000000 -0400 Modify: 2026-06-12 14:15:54.000000000 -0400 Change: 2026-06-12 14:15:54.000000000 -0400 Birth: 2026-06-12 14:15:54.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762658 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:15:54.000000000 -0400 Modify: 2026-06-12 14:15:56.000000000 -0400 Change: 2026-06-12 14:15:56.000000000 -0400 Birth: 2026-06-12 14:15:54.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281062 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:16:01.000000000 -0400 Modify: 2026-06-12 14:16:03.000000000 -0400 Change: 2026-06-12 14:16:03.000000000 -0400 Birth: 2026-06-12 14:16:01.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762660 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:16:01.000000000 -0400 Modify: 2026-06-12 14:16:03.000000000 -0400 Change: 2026-06-12 14:16:03.000000000 -0400 Birth: 2026-06-12 14:16:02.000000000 -0400 PASS 54 (34s) == sanityn test 55a: rename vs unlink target dir ========= 14:16:13 (1781288173) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (11s) == sanityn test 55b: rename vs unlink source dir ========= 14:16:24 (1781288184) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (12s) == sanityn test 55c: rename vs unlink orphan target dir == 14:16:36 (1781288196) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (17s) == sanityn test 55d: rename file vs link ================= 14:16:54 (1781288214) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (14s) == sanityn test 60: Verify data_version behaviour ======== 14:17:08 (1781288228) striped dir -i0 -c2 -H all_char /mnt/lustre/d60.sanityn PASS 60 (7s) == sanityn test 70a: cd directory && rm directory ======== 14:17:15 (1781288235) PASS 70a (6s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 14:17:22 (1781288242) PASS 70b (8s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 14:17:29 (1781288249) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00762041 s, 8.6 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 14:17:33 (1781288253) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0150059 s, 4.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (4s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 14:17:36 (1781288256) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 14:17:39 (1781288259) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0123017 s, 5.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (3s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 14:17:42 (1781288262) llite.lustre-ffff8dfe86913000.xattr_cache=1 llite.lustre-ffff8dfe90fcc800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (7s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 14:17:49 (1781288269) llite.lustre-ffff8dfe86913000.xattr_cache=1 llite.lustre-ffff8dfe90fcc800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (7s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 14:17:56 (1781288276) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0107433 s, 95.3 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.038558 s, 26.6 kB/s 273736: get lock2 273737: get lock1 273736: done 273736: get lock1 273737: done 273737: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 273737: cannot get lock2: Resource deadlock avoided 273737: exit rc=0 273736: done 273736: exit rc=0 PASS 74 (10s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 14:18:06 (1781288286) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0453577 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515854 s, 20.3 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 274319 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (11s) == sanityn test 76: Verify MDT open_files listing ======== 14:18:17 (1781288297) striped dir -i0 -c2 -H crush /mnt/lustre/d76.sanityn opening files: ................ closing files: ................ PASS 76 (160s) == sanityn test 77a: check FIFO NRS policy =============== 14:20:57 (1781288457) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.546025 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095092 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0935774 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120685 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118841 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135015 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105074 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213901 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260958 s, 4.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257731 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.342684 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242808 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276262 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31116 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352501 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.41763 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405477 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112047 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106019 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907566 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093493 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118337 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129515 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122234 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139947 s, 7.5 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112624 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180889 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117616 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146608 s, 7.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128631 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130226 s, 8.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135989 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165209 s, 6.3 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (10s) == sanityn test 77b: check CRR-N NRS policy ============== 14:21:07 (1781288467) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.419904 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423054 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648731 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133211 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169652 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954795 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937861 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689736 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139923 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109749 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201161 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176322 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196473 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23797 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280071 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213908 s, 4.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383335 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0384227 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399962 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045076 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0498885 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531474 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0641978 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979662 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135736 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817122 s, 12.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0902216 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890407 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982691 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985805 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104241 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113313 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134239 s, 7.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.405743 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933941 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873579 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0696928 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136286 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129148 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15848 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18634 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242519 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21496 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20069 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265821 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254746 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237003 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309996 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102209 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321023 s, 3.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130851 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0746265 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00333678 s, 314 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911883 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101065 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136817 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105104 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113217 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147646 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834209 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150317 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616591 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566168 s, 18.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388677 s, 27.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594341 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05338 s, 19.6 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (13s) == sanityn test 77c: check ORR NRS policy ================ 14:21:20 (1781288480) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.439247 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584556 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127339 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118826 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125881 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808003 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175936 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099546 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189971 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18626 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205941 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133662 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252116 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238122 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158512 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179715 s, 5.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254785 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985321 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712753 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610727 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0996495 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936135 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813315 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122048 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123554 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122331 s, 8.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11001 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105219 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10136 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10035 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136636 s, 7.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=6 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128656 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124717 s, 8.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.627625 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04432 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917322 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916507 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220397 s, 4.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920611 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0876638 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243028 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246107 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076621 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186319 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266686 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285351 s, 3.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317356 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.421228 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.437106 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383268 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267478 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129287 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979305 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100955 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121857 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110425 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154043 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112605 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116578 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879851 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163671 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120639 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121275 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0555887 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116259 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129215 s, 8.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.615106 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637383 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536592 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135054 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855226 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171333 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144864 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101846 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462871 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0548186 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238371 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173439 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310829 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207541 s, 5.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201327 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306432 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274526 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644225 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707559 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144942 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499302 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0754631 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775108 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132062 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079812 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700318 s, 15.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957894 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0688336 s, 15.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100564 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135158 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755686 s, 13.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554438 s, 18.9 MB/sldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365967 s, 28.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (18s) == sanityn test 77d: check TRR nrs policy ================ 14:21:38 (1781288498) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.459169 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665725 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612088 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172753 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114917 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121203 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112194 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182638 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218737 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121358 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642228 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23427 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241201 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178638 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292317 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241861 s, 4.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105498 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171352 s, 61.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669047 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941784 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487107 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496751 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551753 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03114 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0614916 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630954 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681482 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0757378 s, 13.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697247 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408258 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706364 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0731098 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819479 s, 12.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.726922 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1485 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219387 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239747 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126253 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226221 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.402333 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271716 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347498 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305872 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276146 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379509 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380611 s, 2.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.351774 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.418045 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309444 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.475779 s, 2.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655045 s, 16.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040595 s, 25.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853168 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282793 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143016 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852596 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856377 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888282 s, 11.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345288 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539011 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105592 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109427 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642476 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966758 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681263 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208752 s, 50.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.421155 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585224 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489943 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853698 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13345 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148459 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154985 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149256 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206698 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17637 s, 5.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240082 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257807 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242029 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256888 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282723 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251284 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322538 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516602 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380796 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336424 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999932 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917161 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362188 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723582 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551576 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044648 s, 23.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571583 s, 18.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956817 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938149 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638757 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551838 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581358 s, 18.0 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368973 s, 28.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (17s) == sanityn test 77e: check TBF NID nrs policy ============ 14:21:56 (1781288516) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.14@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.294939 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108821 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112463 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110473 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126708 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155969 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172357 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131995 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197916 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153626 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190451 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182735 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115912 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244891 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172591 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208438 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285938 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658387 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10402 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0691007 s, 15.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831541 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657065 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877166 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108337 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087848 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856325 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080592 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103446 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082098 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689915 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769144 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070613 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833562 s, 12.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.676656 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097888 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107959 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650983 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15615 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208302 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164336 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144125 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223596 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219402 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179008 s, 5.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251063 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284389 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249643 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210074 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182955 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207948 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139161 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184998 s, 56.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128987 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011468 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153247 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124516 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124771 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109359 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159176 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00256514 s, 409 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100513 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106614 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539984 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332475 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027247 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499385 s, 21.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.494014 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105641 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758438 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127575 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298293 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286773 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260811 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25199 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218809 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340396 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343257 s, 3.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286495 s, 3.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326389 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399634 s, 2.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.39644 s, 2.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.38345 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.446528 s, 2.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0555482 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137191 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136739 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144555 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142812 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684379 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711774 s, 14.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676349 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133667 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162607 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844731 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100437 s, 10.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189468 s, 5.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151677 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126377 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140183 s, 7.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.459198 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082583 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122353 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137059 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156413 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14378 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868758 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138627 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419128 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125527 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215083 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203665 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278726 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229899 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230356 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224141 s, 4.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315903 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268316 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558248 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213623 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00421121 s, 249 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0754333 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047432 s, 22.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718434 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408914 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825731 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417897 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507559 s, 20.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440096 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482558 s, 21.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534232 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0728982 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00367956 s, 285 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=4 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (25s) == sanityn test 77f: check TBF JobID nrs policy ========== 14:22:20 (1781288540) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.430529 s, 39.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0750087 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088416 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105142 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103714 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066509 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121787 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16439 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218753 s, 4.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617187 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183163 s, 5.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223873 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283841 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328262 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.345441 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338566 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.417434 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151061 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0974823 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958544 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114726 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161807 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355598 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616035 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478442 s, 21.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534148 s, 19.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00491783 s, 213 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952712 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00355497 s, 295 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00245722 s, 427 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161917 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636008 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230086 s, 45.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=2 removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.6006 s, 27.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121279 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114907 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961019 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683324 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114085 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177866 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328494 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282898 s, 3.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309962 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302917 s, 3.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222288 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31912 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28171 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329205 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357862 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.413326 s, 2.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732042 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349407 s, 30.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724753 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650456 s, 16.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072337 s, 14.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170079 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132574 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118241 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106014 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111747 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00512262 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205375 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066361 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779291 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0528377 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178888 s, 5.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.601093 s, 27.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549102 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833756 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817659 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712527 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0972168 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151187 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104576 s, 10.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164943 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105259 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352249 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376148 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372767 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.407245 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00245462 s, 427 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950867 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3783 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.466079 s, 2.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628484 s, 167 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151028 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00230978 s, 454 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.493049 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748109 s, 140 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273554 s, 3.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192029 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297591 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737979 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505355 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015688 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745846 s, 14.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509301 s, 20.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425329 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355779 s, 29.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.471875 s, 35.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045114 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092225 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111863 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330732 s, 31.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124512 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1122 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157605 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189899 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27988 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281377 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335496 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816629 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102465 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243975 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529121 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338163 s, 3.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720048 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800704 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136101 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663825 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337439 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697941 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375249 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444454 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605237 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509939 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579954 s, 18.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060762 s, 17.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0636046 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103684 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933293 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131728 s, 8.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (28s) == sanityn test 77g: Change TBF type directly ============ 14:22:48 (1781288568) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (14s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 14:23:02 (1781288582) oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg314-client: oleg314-server: ssh exited with exit code 2 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 12 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 12 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 12 PASS 77h (14s) == sanityn test 77i: Change rank of TBF rule ============= 14:23:16 (1781288596) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (24s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 14:23:40 (1781288620) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.533075 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517219 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075683 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131923 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111368 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150611 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157799 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149833 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145378 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179472 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135057 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183261 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217713 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244897 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254828 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257867 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238776 s, 4.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319392 s, 32.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360361 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445186 s, 23.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517652 s, 20.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371345 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372102 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860974 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431807 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618259 s, 17.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466749 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527307 s, 19.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610546 s, 17.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619672 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061955 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676984 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775023 s, 13.5 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1956 s, 10.3 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5894 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (69s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 14:24:49 (1781288689) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.86359 s, 19.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0818896 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524211 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11139 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140692 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109656 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109972 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713188 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568301 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241284 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183066 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195033 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268222 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344295 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265041 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314191 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.377474 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078021 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557992 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104199 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559998 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677698 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643397 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693294 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524713 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720122 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862023 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085814 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905554 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948974 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10233 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129228 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126256 s, 8.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9315 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0014 s, 5.2 MB/s Read runtime is 22 s, speed is 4.545454 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.907662 s, 18.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146002 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168135 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110994 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323217 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194806 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305433 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319364 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187124 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288663 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266631 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349826 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.440022 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297198 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.468213 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254639 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347201 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712561 s, 14.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966335 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771862 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694243 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919079 s, 11.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083592 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836554 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678852 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870732 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823631 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108958 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844903 s, 12.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952312 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104677 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943613 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104923 s, 10.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.168 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8447 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (154s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 14:27:23 (1781288843) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.447055 s, 37.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835496 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844151 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103954 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215652 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316966 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297502 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313025 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29793 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.348699 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361273 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260307 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347933 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309885 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283868 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31371 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416933 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299933 s, 35.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072459 s, 14.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131667 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00371033 s, 283 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00255142 s, 411 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102897 s, 102 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153854 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618687 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100865 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816876 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0818667 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861085 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424275 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357874 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323961 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306753 s, 34.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75328 s, 10.8 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.97721 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.14@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.14@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.43205 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743062 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10557 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110405 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215206 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256218 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258012 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205192 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227875 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243257 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28422 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318487 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234356 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280845 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263294 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29723 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26478 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395543 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047257 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713366 s, 14.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102726 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130768 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562129 s, 18.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066986 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534771 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077299 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822213 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644145 s, 16.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823273 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727511 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798255 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165368 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135103 s, 77.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.26556 s, 12.7 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.86609 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.14@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.402917 s, 41.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100943 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907955 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117703 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136684 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151877 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609129 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199396 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1766 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101295 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151373 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287778 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129988 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240758 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321745 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229103 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400099 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126142 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648824 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196185 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125289 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120657 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133459 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078607 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100033 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993766 s, 10.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1535 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108531 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105562 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159694 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119966 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164002 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172592 s, 6.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.84284 s, 13.4 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.05145 s, 20.8 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.57876 s, 29.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416509 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811058 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107156 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184081 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207192 s, 5.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219444 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178819 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291322 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278257 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260402 s, 4.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293642 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277596 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373116 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343236 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.398528 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.401533 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238555 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0040871 s, 257 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642655 s, 163 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907958 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00431845 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760894 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00401753 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00505829 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603774 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133229 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110094 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00539313 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121466 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103922 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0598461 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030006 s, 34.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.965 s, 8.8 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.95166 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.9146 s, 9.6 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9221 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.634838 s, 26.4 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715208 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379704 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120528 s, 8.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122462 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163393 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129116 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179427 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111218 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175019 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171695 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277716 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287274 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277322 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.370053 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285805 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284648 s, 3.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376439 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00427011 s, 246 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906794 s, 11.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0545066 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014356 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135013 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00286983 s, 365 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160789 s, 65.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242439 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819793 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207027 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01071 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106742 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559332 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562021 s, 18.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9947 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6277 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.514953 s, 32.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638295 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919255 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874205 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112242 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137241 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210387 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295715 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250752 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330872 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331824 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311912 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357771 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343414 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340737 s, 3.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.422165 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.360296 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292118 s, 35.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389197 s, 26.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577693 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420131 s, 25.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114436 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507135 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0721711 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562028 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816774 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832948 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878091 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983778 s, 10.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127884 s, 8.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999147 s, 10.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808172 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160304 s, 6.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.24937 s, 12.7 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9234 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.69728 s, 24.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442052 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815073 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952898 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242895 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243639 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.307224 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294792 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269006 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29523 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325958 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.289871 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326756 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260442 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.348723 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356496 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365039 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339443 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786222 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745073 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447764 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04228 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549152 s, 19.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0633679 s, 16.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078685 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651651 s, 16.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690344 s, 15.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594037 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073383 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772422 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762046 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074478 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762919 s, 13.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.5395 s, 9.1 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.93325 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.8672 s, 8.8 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91794 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (441s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 14:34:44 (1781289284) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (11s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 14:34:56 (1781289296) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg314-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.93659 s, 266 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg314-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0963 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (58s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 14:35:55 (1781289355) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.465823 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566444 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575587 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117796 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17335 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832015 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170716 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676094 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131229 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0679025 s, 15.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102684 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170222 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215881 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292599 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303042 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356786 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.435299 s, 2.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447986 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287692 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393333 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795489 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152723 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0430314 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617502 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566584 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582352 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635466 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830133 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878693 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893343 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926779 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102795 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976077 s, 10.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.56248 s, 12.2 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.4353 s, 19.3 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.419534 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133497 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137798 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182182 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770826 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978707 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222856 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100924 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202529 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205716 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210491 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173623 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162204 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799777 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249704 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24931 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244267 s, 4.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8dfe86913000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659831 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270039 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.064415 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062305 s, 16.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385212 s, 27.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517476 s, 20.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385493 s, 27.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395182 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442507 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851945 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806283 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825105 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0501513 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569674 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551766 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0589347 s, 17.8 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73259 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.31738 s, 19.7 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (95s) == sanityn test 77o: Changing rank should not panic ====== 14:37:30 (1781289450) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (15s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 14:37:45 (1781289465) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (109s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 14:39:34 (1781289574) mds.MDS.mdt.nrs_policies=tbf oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg314-client: oleg314-server: ssh exited with exit code 36 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (34s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 14:40:09 (1781289609) oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg314-client: oleg314-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (11s) == sanityn test 79: xattr: intent error ================== 14:40:20 (1781289620) striped dir -i1 -c2 -H all_char /mnt/lustre/d79.sanityn File: /mnt/lustre/d79.sanityn Size: 12288 Blocks: 24 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762977 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:40:22.000000000 -0400 Modify: 2026-06-12 14:40:22.000000000 -0400 Change: 2026-06-12 14:40:22.000000000 -0400 Birth: 2026-06-12 14:40:22.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (19s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 14:40:39 (1781289639) total: 10 open/close in 0.19 seconds: 53.48 ops/second multiop /mnt/lustre2/d80a.sanityn/f80a.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 lfs migrate: /mnt/lustre/d80a.sanityn/f80a.sanityn migrate failed: Device or resource busy (16) Finish migration, then checking.. PASS 80a (15s) == sanityn test 80b: Accessing directory during migration ========================================================== 14:40:54 (1781289654) start migration thread 301417 accessing the migrating directory for 1 minute... concurrent access 17 failures, 472 successes /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 301417 Killed ( while true; do mdt_idx=$((RANDOM % MDSCOUNT)); $LFS migrate -m $mdt_idx $migdir1 &> /dev/null; done ) (wd: ~) PASS 80b (69s) == sanityn test 81a: rename and stat under striped directory ========================================================== 14:42:03 (1781289723) stat: cannot statx 'd0/0': No such file or directory File: d0/0 Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289281626 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:42:05.000000000 -0400 Modify: 2026-06-12 14:42:05.000000000 -0400 Change: 2026-06-12 14:42:05.000000000 -0400 Birth: 2026-06-12 14:42:05.000000000 -0400 PASS 81a (8s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 14:42:11 (1781289731) - open/close 670 (time 1781289745.02 total 10.01 last 66.94) total: 1000 open/close in 14.73 seconds: 67.90 ops/second /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 303310 Killed ( while true; do touch $DIR1/$tdir; done ) (wd: ~) PASS 81b (188s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 14:45:19 (1781289919) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (3s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 14:45:22 (1781289922) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7880 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7880 PASS 82 (7s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 14:45:29 (1781289929) start pid 306524 to create/unlink striped directory start pid 306525 to stat striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4949: 306524 Terminated ( cd $DIR1; while true; do $LFS mkdir -i1 -c2 $tdir; rmdir $tdir; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4949: 306525 Terminated ( cd $DIR2; while true; do stat $tdir > /dev/null 2>&1; done ) PASS 83 (130s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 14:47:39 (1781290059) fail_loc=0x8000060b PASS 84 (13s) == sanityn test 90: open/create and unlink striped directory ========================================================== 14:47:52 (1781290072) start pid 308086 to open/create under striped directory start pid 308087 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5011: 308086 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; touch $tdir/f{0..3} > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5011: 308087 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 90 (187s) == sanityn test 91: chmod and unlink striped directory === 14:50:59 (1781290259) start pid 309745 to chmod striped directory start pid 309746 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5048: 309745 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; chmod go+w $tdir > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5048: 309746 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 91 (187s) == sanityn test 92: create remote directory under orphan directory ========================================================== 14:54:06 (1781290446) fail_loc=0x1408 mkdir: cannot create directory '/mnt/lustre2/d92.sanityn/dir': No such file or directory lfs setdirstripe: cannot resolve path '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory (2) lfs setdirstripe: '/mnt/lustre2/d92.sanityn/remote_dir' is not on a Lustre filesystem: No such file or directory (2) lfs setdirstripe: cannot create dir '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory fail_loc=0 PASS 92 (6s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 14:54:12 (1781290452) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1804 0x70c 0 0 1811 0x713 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1805 0x70d 0 0 1812 0x714 0 rc1=0 and rc2=0 PASS 93 (15s) == sanityn test 94: signal vs CP callback race =========== 14:54:27 (1781290467) osc.lustre-OST0000-osc-ffff8dfe86913000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8dfe90fcc800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8dfe86913000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8dfe90fcc800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.239283 s, 34.2 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 2.99711 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 312997 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0625158 s, 65.5 kB/s osc.lustre-OST0000-osc-ffff8dfe86913000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8dfe90fcc800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8dfe86913000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8dfe90fcc800.idle_timeout=debug PASS 94 (14s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 14:54:42 (1781290482) llite.lustre-ffff8dfe86913000.fast_read=0 llite.lustre-ffff8dfe90fcc800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00800203 s, 1.5 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8dfe86913000.fast_read=1 llite.lustre-ffff8dfe90fcc800.fast_read=1 PASS 95b (13s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 14:54:54 (1781290494) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 14:54:57 (1781290497) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00440892 s, 929 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (7s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 14:55:04 (1781290504) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.135278 s, 15.5 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (6s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 14:55:10 (1781290510) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0872598 s, 24.0 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (5s) == sanityn test 100e: DoM: read on open and file size ==== 14:55:16 (1781290516) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00633409 s, 1.9 MB/s PASS 100e (5s) == sanityn test 101a: Discard DoM data on unlink ========= 14:55:22 (1781290522) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00990905 s, 413 kB/s PASS 101a (7s) == sanityn test 101b: Discard DoM data on rename ========= 14:55:28 (1781290528) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00897943 s, 456 kB/s PASS 101b (6s) == sanityn test 101c: Discard DoM data on close-unlink === 14:55:34 (1781290534) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00516224 s, 793 kB/s PASS 101c (8s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 14:55:44 (1781290544) striped dir -i1 -c2 -H crush /mnt/lustre/d103.sanityn fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1781290547 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1781290550 Finishing test test23 at 1781290551 PASS 103 (15s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 14:55:59 (1781290559) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (4s) == sanityn test 105: Glimpse and lock cancel race ======== 14:56:03 (1781290563) striped dir -i1 -c2 -H crush /mnt/lustre/d105.sanityn fail_loc=0x416 PASS 105 (84s) == sanityn test 106a: Verify the btime via statx() ======= 14:57:28 (1781290648) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (3s) == sanityn test 106b: Glimpse RPCs test for statx ======== 14:57:31 (1781290651) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0161641 s, 63.4 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289282505 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:57:33.000000000 -0400 Modify: 2026-06-12 14:57:33.000000000 -0400 Change: 2026-06-12 14:57:33.000000000 -0400 Birth: 2026-06-12 14:57:33.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205289282505 -rw-r--r-- 1 0 0 1781290653 1781290653 1781290653 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289282505 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:57:33.000000000 -0400 Modify: 2026-06-12 14:57:33.000000000 -0400 Change: 2026-06-12 14:57:33.000000000 -0400 Birth: 2026-06-12 14:57:33.000000000 -0400 1781290653 1024 1 PASS 106b (7s) == sanityn test 106c: Verify statx attributes mask ======= 14:57:38 (1781290658) PASS 106c (6s) == sanityn test 107a: Basic grouplock conflict =========== 14:57:44 (1781290664) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.215917 s, 48.6 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 1816 0x718 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 First grouplock blocks second one PASS 107a (9s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 14:57:53 (1781290673) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.233893 s, 44.8 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1811 0x713 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7880 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (13s) == sanityn test 108a: lseek: parallel updates ============ 14:58:06 (1781290686) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0162777 s, 614 kB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.78516 s, 1.8 MB/s PASS 108a (11s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 14:58:17 (1781290697) 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 6 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 PASS 109 (1354s) == sanityn test 110: do not grant another lock on resend ========================================================== 15:20:51 (1781292051) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934275 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 15:20:53.000000000 -0400 Modify: 2026-06-12 15:20:53.000000000 -0400 Change: 2026-06-12 15:20:53.000000000 -0400 Birth: 2026-06-12 15:20:53.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg314-client.virtnet oleg314-client.virtnet PASS 110 (44s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 15:21:36 (1781292096) fail_loc=0x8000018a /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5892: 382312 Terminated $MULTIOP $DIR2/$tdir/mdt0dir/foodir/file2 Ow4096_c PASS 111 (15s) == sanityn test 112: update max-inherit in default LMV === 15:21:50 (1781292110) lmv.lustre-clilmv-ffff8dfe90fcb800.qos_threshold_rr=90 lmv.lustre-clilmv-ffff8dfe93bd2000.qos_threshold_rr=90 PASS 112 (16s) == sanityn test 113: check servers of specified fs ======= 15:22:06 (1781292126) lustre-OST0000-osc-ffff8dfe90fcb800 active. lustre-OST0001-osc-ffff8dfe90fcb800 active. lustre-OST0000-osc-ffff8dfe90fcb800 active. lustre-OST0001-osc-ffff8dfe90fcb800 active. MGC192.168.203.114@tcp active. PASS 113 (8s) cleanup: ====================================================== == sanityn test complete, duration 11252 sec ============= 15:22:16 (1781292136) Stopping clients: oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 opts: