-----============= acceptance-small: sanityn ============----- Mon May 11 17:26:28 EDT 2026 oleg420-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Started clients oleg420-client.virtnet: 192.168.204.120@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg420-client.virtnet: executing check_config_client /mnt/lustre oleg420-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg420-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff915f42c40000.idle_timeout=debug osc.lustre-OST0000-osc-ffff915f457d0800.idle_timeout=debug osc.lustre-OST0001-osc-ffff915f42c40000.idle_timeout=debug osc.lustre-OST0001-osc-ffff915f457d0800.idle_timeout=debug disable quota as required oleg420-server: oleg420-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200259 s, 52.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7129] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 17:26:57 (1778534817) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (4s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 17:27:01 (1778534821) -rw-r--r-- 1 root root 0 May 11 17:27 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 17:27:05 (1778534825) -rw-r--r-- 1 root root 0 May 11 17:27 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 17:27:09 (1778534829) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 17:27:14 (1778534834) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 17:27:17 (1778534837) total 1 -rwxrwxrwx 1 root root 0 May 11 17:27 f2a -rwxrwxrwx 1 root root 0 May 11 17:27 f2b total 1 -rwxrwxrwx 1 root root 0 May 11 17:27 f2a -rwxrwxrwx 1 root root 0 May 11 17:27 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00562498 s, 91.0 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 17:27:21 (1778534841) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 17:27:23 (1778534843) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.250893 s, 8.4 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 17:27:27 (1778534847) PASS 3 (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 17:27:31 (1778534851) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 17:27:36 (1778534856) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (4s) == sanityn test 6: remove of open file on other node ============================================================================ 17:27:40 (1778534860) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 17:27:45 (1778534865) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (5s) == sanityn test 8: remove of open special file on other node ==================================================================== 17:27:49 (1778534869) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 17:27:53 (1778534873) PASS 9a (5s) == sanityn test 9b: append to striped sparse file ======== 17:27:58 (1778534878) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572428 s, 18.3 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0447802 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (4s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 17:28:02 (1778534882) 1+0 records in 1+0 records out 1 byte copied, 0.00644892 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0107531 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0179035 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00458536 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0127765 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00570255 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00453779 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00419631 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00621869 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.003059 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0040296 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0056594 s, 0.2 kB/s PASS 10a (5s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 17:28:07 (1778534887) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0149089 s, 206 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0416766 s, 98.3 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000595115 s, 5.2 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 17:28:11 (1778534891) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (5s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 17:28:16 (1778534896) warning: '-runtime' deprecated, use '-t runtime' instead - link 604 (time 1778534907.65 total 10.01 last 60.36) - link 1167 (time 1778534917.65 total 20.01 last 56.29) - link 1765 (time 1778534927.65 total 30.02 last 59.76) - link 2340 (time 1778534937.67 total 40.03 last 57.40) - link 2913 (time 1778534947.68 total 50.04 last 57.24) - link 3435 (time 1778534957.70 total 60.07 last 52.08) - link 4013 (time 1778534967.72 total 70.08 last 57.74) - link 4570 (time 1778534977.73 total 80.09 last 55.65) - link 5125 (time 1778534987.74 total 90.10 last 55.45) total: 5668 link in 100.01 seconds: 56.67 ops/second using seed 1708044254 running for 100 seconds - stat 10000 (time 1778534905 ; total 7 ; last 7) - stat 20000 (time 1778534913 ; total 15 ; last 8) - stat 30000 (time 1778534921 ; total 23 ; last 8) - stat 40000 (time 1778534928 ; total 30 ; last 7) - stat 50000 (time 1778534935 ; total 37 ; last 7) - stat 60000 (time 1778534943 ; total 45 ; last 8) - stat 69991 (time 1778534954 ; total 56 ; last 11) - stat 70000 (time 1778534954 ; total 56 ; last 0) - stat 76689 (time 1778534965 ; total 67 ; last 11) - stat 80000 (time 1778534969 ; total 71 ; last 4) - stat 86490 (time 1778534980 ; total 82 ; last 11) - stat 90000 (time 1778534985 ; total 87 ; last 5) - stat 97156 (time 1778534996 ; total 98 ; last 11) total: 98952 stats in 100 seconds: 989.520020 stats/second - unlinked 0 (time 1778534948 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile5668) error: No such file or directory total: 5667 unlinks in 85 seconds: 66.670586 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19692) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19695) - No such process PASS 12 (140s) == sanityn test 13: test directory page revocation ======= 17:30:36 (1778535036) f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 17:30:41 (1778535041) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20866 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (3s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 17:30:44 (1778535044) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21440 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (4s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 17:30:48 (1778535048) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22014 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 17:30:51 (1778535051) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22589 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 17:30:55 (1778535055) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23165 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 17:30:59 (1778535059) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (15s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 17:31:14 (1778535074) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.194344 s, 10.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.132311 s, 15.9 MB/s Chance of close/open is 1 in 50 Seed set to 5076 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x291f6e 000001[0] 1778535076.671357 trunc from 00000000 to 0x291f6d (0x291f6e bytes) 000004[1] 1778535076.693653 mapwrite 0x83ed15 thru 0x846e91 (0x817d bytes) truncating to largest ever: 0x3cbd59 000006[1] 1778535076.794438 trunc from 0x846e92 to 0x3cbd58 (0x47b139 bytes) truncating to largest ever: 0x82ab73 000007[1] 1778535076.808512 trunc from 0x3cbd59 to 0x82ab72 (0x45ee1a bytes) 000008[0] 1778535076.822160 write 0x4f62f2 thru 0x500260 (0x9f6f bytes) 000010[0] 1778535076.869121 mapwrite 0x13f615 thru 0x148937 (0x9323 bytes) truncating to largest ever: 0x9a1899 000014[1] 1778535076.983163 trunc from 0x82ab73 to 0x9a1898 (0x176d26 bytes) 000017[0] 1778535077.039950 mapread 0x74586c thru 0x746cc6 (0x145b bytes) 000022[1] 1778535077.109986 mapread 0x4f83d7 thru 0x4f97b7 (0x13e1 bytes) 000023[0] 1778535077.133185 read 0x7947a6 thru 0x795a1e (0x1279 bytes) 000026[1] 1778535077.157646 mapwrite 0x446f99 thru 0x44beea (0x4f52 bytes) 000028[0] 1778535077.214481 mapwrite 0x1dc93c thru 0x1ebcd4 (0xf399 bytes) 000031[1] 1778535077.313185 mapwrite 0x277136 thru 0x27aac5 (0x3990 bytes) 000032[0] 1778535077.388604 write 0x40c8e5 thru 0x41b2cf (0xe9eb bytes) 000033[1] 1778535077.426974 read 0x1b6f39 thru 0x1be313 (0x73db bytes) 000034[1] 1778535077.468721 mapread 0x57ee5b thru 0x5853b1 (0x6557 bytes) 000036[0] 1778535077.501394 read 0x00da47 thru 0x0154d2 (0x7a8c bytes) 000037[0] 1778535077.576279 trunc from 0x9a1899 to 0x018016 (0x989882 bytes) 000038[1] 1778535077.650082 trunc from 0x018017 to 0x08abb9 (0x72ba3 bytes) 000039[0] 1778535077.686399 trunc from 0x08abba to 0x8b28d9 (0x827d20 bytes) 000041[1] 1778535077.737628 mapread 0x04f857 thru 0x05da7b (0xe225 bytes) 000043[1] 1778535077.778851 read 0x3c4753 thru 0x3cf75a (0xb008 bytes) 000044[0] 1778535077.840532 mapwrite 0x531185 thru 0x53d1a5 (0xc021 bytes) 000049[1] 1778535077.986639 mapread 0x4b8232 thru 0x4c38b0 (0xb67f bytes) 000051[0] 1778535077.992974 trunc from 0x8b28da to 0x5e664e (0x2cc28b bytes) 000053[1] 1778535078.018619 trunc from 0x5e664f to 0x149c55 (0x49c9f9 bytes) 000054[1] 1778535078.057889 trunc from 0x149c56 to 0x5c455f (0x47a90a bytes) 000055[1] 1778535078.075345 write 0x409403 thru 0x4125de (0x91dc bytes) 000059[0] 1778535078.127576 write 0x101540 thru 0x106fee (0x5aaf bytes) 000062[1] 1778535078.168876 mapread 0x4d3a90 thru 0x4de783 (0xacf4 bytes) 000068[0] 1778535078.277466 trunc from 0x5c4560 to 0x98cfc1 (0x3c8a62 bytes) 000070[1] 1778535078.331088 trunc from 0x98cfc2 to 0x42c3c6 (0x560bfb bytes) truncating to largest ever: 0x9ea292 000071[0] 1778535078.366234 trunc from 0x42c3c7 to 0x9ea291 (0x5bdecb bytes) 000072[0] 1778535078.386746 mapwrite 0x9ad343 thru 0x9b4967 (0x7625 bytes) 000073[1] 1778535078.428042 mapwrite 0x99c65d thru 0x9a6986 (0xa32a bytes) 000074[1] 1778535078.525803 trunc from 0x9ea292 to 0x371fbe (0x6782d3 bytes) 000075[1] 1778535078.561685 mapwrite 0x51b66b thru 0x51f8f8 (0x428e bytes) 000078[1] 1778535078.656742 write 0x0671e4 thru 0x076f72 (0xfd8f bytes) 000080[0] 1778535078.684102 mapwrite 0x72e492 thru 0x733c83 (0x57f2 bytes) 000081[0] 1778535078.783473 mapwrite 0x607cfd thru 0x60e939 (0x6c3d bytes) 000083[0] 1778535078.862640 read 0x3093cf thru 0x30e338 (0x4f6a bytes) 000084[1] 1778535078.908004 trunc from 0x733c84 to 0x79a27a (0x665f7 bytes) 000086[1] 1778535078.963255 mapwrite 0x1d7114 thru 0x1e3cef (0xcbdc bytes) 000089[0] 1778535079.065498 mapwrite 0x90c1f2 thru 0x919a8c (0xd89b bytes) 000090[1] 1778535079.183965 trunc from 0x919a8d to 0x67bdfe (0x29dc8e bytes) 000092[0] 1778535079.256023 mapwrite 0x79fc8e thru 0x7af935 (0xfca8 bytes) 000095[0] 1778535079.408357 write 0x2414c3 thru 0x248a72 (0x75b0 bytes) 000099[1] 1778535079.456875 write 0x043d9c thru 0x0520df (0xe344 bytes) 000100[0] 1778535079.483872 mapread 0x314b41 thru 0x319a0e (0x4ece bytes) 000101[1] 1778535079.514682 read 0x63ba2a thru 0x64650b (0xaae2 bytes) 000102[0] 1778535079.541560 write 0x6ce4d6 thru 0x6dd919 (0xf444 bytes) 000103[1] 1778535079.569000 mapwrite 0x7b076a thru 0x7b920c (0x8aa3 bytes) 000104[1] 1778535079.683536 trunc from 0x7b920d to 0x217cea (0x5a1522 bytes) 000106[0] 1778535079.746873 write 0x2461d1 thru 0x250fd6 (0xae06 bytes) 000107[1] 1778535079.771320 write 0x3377e8 thru 0x33ae3d (0x3656 bytes) 000108[0] 1778535079.789599 mapread 0x07c0d1 thru 0x0844f9 (0x8429 bytes) 000109[0] 1778535079.834446 read 0x089e04 thru 0x09950a (0xf707 bytes) 000112[0] 1778535079.897567 mapwrite 0x4b2d90 thru 0x4c0ad9 (0xdd4a bytes) 000113[0] 1778535080.046629 write 0x958b4b thru 0x96638e (0xd844 bytes) 000114[0] 1778535080.062828 write 0x8d692a thru 0x8d75c8 (0xc9f bytes) 000116[1] 1778535080.092929 mapwrite 0x8d90c5 thru 0x8df52b (0x6467 bytes) 000118[1] 1778535080.285872 mapread 0x514875 thru 0x516091 (0x181d bytes) 000120[1] 1778535080.324571 write 0x69d60c thru 0x6a2edc (0x58d1 bytes) 000121[1] 1778535080.344199 read 0x0933f8 thru 0x093bdd (0x7e6 bytes) 000122[0] 1778535080.407136 write 0x20915a thru 0x20a4c7 (0x136e bytes) 000125[1] 1778535080.435569 read 0x328256 thru 0x33078e (0x8539 bytes) 000126[0] 1778535080.436225 mapwrite 0x8b1e7b thru 0x8b5fb1 (0x4137 bytes) 000128[1] 1778535080.505870 read 0x92840c thru 0x92e75b (0x6350 bytes) 000129[1] 1778535080.506639 read 0x74a34e thru 0x759024 (0xecd7 bytes) 000130[1] 1778535080.507510 mapwrite 0x7386d7 thru 0x73938c (0xcb6 bytes) 000131[0] 1778535080.580316 write 0x0d024e thru 0x0d69c8 (0x677b bytes) 000132[0] 1778535080.601817 read 0x669919 thru 0x673342 (0x9a2a bytes) 000136[1] 1778535080.652494 read 0x946192 thru 0x94db63 (0x79d2 bytes) 000137[1] 1778535080.683208 mapread 0x1394da thru 0x146ce5 (0xd80c bytes) 000140[1] 1778535080.778585 mapwrite 0x1e2061 thru 0x1ea9fd (0x899d bytes) 000141[1] 1778535080.881109 read 0x051423 thru 0x060a23 (0xf601 bytes) 000142[0] 1778535080.975325 mapwrite 0x392081 thru 0x392ee6 (0xe66 bytes) 000143[0] 1778535081.069530 write 0x7b690d thru 0x7c3833 (0xcf27 bytes) 000147[1] 1778535081.084393 mapwrite 0x9387dc thru 0x942fd8 (0xa7fd bytes) 000149[1] 1778535081.206573 write 0x231dee thru 0x23c787 (0xa99a bytes) 000150[0] 1778535081.240099 mapwrite 0x1dd8be thru 0x1dff55 (0x2698 bytes) 000151[0] 1778535081.312050 trunc from 0x96638f to 0x7f4d57 (0x171637 bytes) 000152[0] 1778535081.337037 write 0x7cb370 thru 0x7d143a (0x60cb bytes) 000153[1] 1778535081.344015 mapread 0x5d6dcc thru 0x5e5526 (0xe75b bytes) 000154[0] 1778535081.405022 mapwrite 0x5b5a23 thru 0x5c4415 (0xe9f3 bytes) 000155[1] 1778535081.501233 write 0x015ec0 thru 0x0162a7 (0x3e8 bytes) 000156[1] 1778535081.531926 mapwrite 0x0d1129 thru 0x0d4c37 (0x3b0f bytes) 000157[0] 1778535081.601422 trunc from 0x7f4d58 to 0x1ed3c3 (0x607994 bytes) 000158[1] 1778535081.626479 write 0x609afd thru 0x614689 (0xab8d bytes) 000159[0] 1778535081.653593 trunc from 0x61468a to 0x3460be (0x2ce5cb bytes) 000163[0] 1778535081.745890 read 0x2aaf93 thru 0x2adb86 (0x2bf4 bytes) 000165[1] 1778535081.765831 mapread 0x340fe8 thru 0x3460be (0x50d7 bytes) 000166[0] 1778535081.808013 mapwrite 0x6b2682 thru 0x6bf2f3 (0xcc72 bytes) 000168[0] 1778535081.917966 mapwrite 0x952b5e thru 0x955d65 (0x3208 bytes) 000170[1] 1778535082.025279 trunc from 0x955d66 to 0x5c1625 (0x394740 bytes) 000171[1] 1778535082.071887 trunc from 0x5c1626 to 0x602d89 (0x41764 bytes) 000172[0] 1778535082.098854 read 0x277f84 thru 0x27a40a (0x2487 bytes) 000176[0] 1778535082.149428 write 0x776f59 thru 0x779a77 (0x2b1f bytes) 000177[1] 1778535082.177651 write 0x7bcabe thru 0x7c3acc (0x700f bytes) 000178[1] 1778535082.200725 trunc from 0x7c3acd to 0x29659b (0x52d531 bytes) 000179[0] 1778535082.257733 write 0x20c1e0 thru 0x212450 (0x6271 bytes) 000182[0] 1778535082.316776 mapread 0x0d461a thru 0x0e218d (0xdb74 bytes) 000183[0] 1778535082.383666 write 0x6d9bc0 thru 0x6e0c89 (0x70ca bytes) 000184[1] 1778535082.398991 mapread 0x5ef82c thru 0x5f247d (0x2c52 bytes) 000185[0] 1778535082.476914 mapread 0x4c929a thru 0x4cad1b (0x1a82 bytes) 000187[1] 1778535082.520966 write 0x1b64e4 thru 0x1b9af4 (0x3611 bytes) 000188[0] 1778535082.541850 mapwrite 0x45f50b thru 0x46f1d3 (0xfcc9 bytes) 000190[1] 1778535082.688294 mapwrite 0x225883 thru 0x235326 (0xfaa4 bytes) 000192[1] 1778535082.849856 write 0x903b8a thru 0x9058cd (0x1d44 bytes) 000193[0] 1778535082.851094 write 0x87773c thru 0x8786b6 (0xf7b bytes) 000194[1] 1778535082.865359 write 0x485a9c thru 0x48a9b9 (0x4f1e bytes) 000195[1] 1778535082.883932 mapread 0x8be46a thru 0x8c16ee (0x3285 bytes) 000196[1] 1778535082.897755 mapread 0x89356e thru 0x8a2e55 (0xf8e8 bytes) 000197[0] 1778535082.961345 write 0x2e7aeb thru 0x2eaa3e (0x2f54 bytes) 000198[0] 1778535082.998922 trunc from 0x9058ce to 0x6614ed (0x2a43e0 bytes) 000199[0] 1778535083.028686 trunc from 0x6614ee to 0x1cec0f (0x4928de bytes) 000200[1] 1778535083.042008 write 0x2066f7 thru 0x212e2f (0xc739 bytes) 000202[1] 1778535083.071302 read 0x02ca53 thru 0x0380ee (0xb69c bytes) 000204[0] 1778535083.111229 trunc from 0x212e30 to 0x89e1cd (0x68b39e bytes) 000205[0] 1778535083.165498 mapread 0x7cbd0b thru 0x7cf6df (0x39d5 bytes) 000209[1] 1778535083.261119 write 0x5a8749 thru 0x5b15fe (0x8eb6 bytes) 000211[1] 1778535083.319931 mapwrite 0x6af869 thru 0x6b2471 (0x2c09 bytes) 000212[0] 1778535083.433974 mapwrite 0x21bcc2 thru 0x2264f3 (0xa832 bytes) 000214[1] 1778535083.583747 mapwrite 0x6cf81d thru 0x6dc2fb (0xcadf bytes) 000215[1] 1778535083.700500 read 0x541ebc thru 0x54d2fa (0xb43f bytes) 000218[1] 1778535083.702925 mapread 0x7282b4 thru 0x72895b (0x6a8 bytes) 000219[1] 1778535083.722044 mapread 0x5d8372 thru 0x5e2ab5 (0xa744 bytes) 000221[1] 1778535083.746054 mapwrite 0x0e1b53 thru 0x0e3c37 (0x20e5 bytes) 000223[1] 1778535083.808156 write 0x2f488e thru 0x2fcfd2 (0x8745 bytes) 000225[0] 1778535083.826845 mapread 0x47d992 thru 0x48344c (0x5abb bytes) 000226[1] 1778535083.891817 trunc from 0x89e1ce to 0x97c03f (0xdde72 bytes) 000227[0] 1778535083.927532 mapwrite 0x64f5f0 thru 0x65c293 (0xcca4 bytes) 000228[1] 1778535084.083025 mapread 0x02a4a1 thru 0x037003 (0xcb63 bytes) 000229[0] 1778535084.167237 mapread 0x235dee thru 0x237697 (0x18aa bytes) 000230[1] 1778535084.182654 write 0x1bc5a6 thru 0x1c6033 (0x9a8e bytes) 000233[1] 1778535084.209145 mapread 0x19c122 thru 0x1aa032 (0xdf11 bytes) 000234[0] 1778535084.220597 mapwrite 0x1b4675 thru 0x1c3335 (0xecc1 bytes) 000236[0] 1778535084.351543 read 0x4b2f9b thru 0x4b50fe (0x2164 bytes) 000237[1] 1778535084.416902 write 0x87651e thru 0x88365b (0xd13e bytes) 000241[1] 1778535084.495218 mapwrite 0x36a645 thru 0x36e926 (0x42e2 bytes) 000242[0] 1778535084.585286 trunc from 0x97c040 to 0x4b6b47 (0x4c54f8 bytes) 000243[1] 1778535084.642344 read 0x3039f6 thru 0x31188f (0xde9a bytes) 000244[1] 1778535084.676288 trunc from 0x4b6b48 to 0x6b2a59 (0x1fbf12 bytes) 000246[1] 1778535084.721412 mapwrite 0x0a63b2 thru 0x0b13a5 (0xaff4 bytes) 000247[1] 1778535084.853112 trunc from 0x6b2a5a to 0x86af2b (0x1b84d2 bytes) 000248[0] 1778535084.874467 mapwrite 0x9f56c6 thru 0x9fc00d (0x6948 bytes) 000250[0] 1778535084.993448 mapread 0x2183cc thru 0x21daa4 (0x56d9 bytes) 000257[0] 1778535085.058738 read 0x8aa038 thru 0x8b71a5 (0xd16e bytes) 000258[1] 1778535085.069030 mapwrite 0x176977 thru 0x179326 (0x29b0 bytes) 000259[0] 1778535085.154880 read 0x1765b0 thru 0x179c6f (0x36c0 bytes) 000261[1] 1778535085.256992 trunc from 0x9fc00e to 0x15bdb9 (0x8a0254 bytes) 000262[0] 1778535085.316011 trunc from 0x15bdba to 0x8d4b83 (0x778dca bytes) 000264[1] 1778535085.389473 trunc from 0x8d4b84 to 0x7359c9 (0x19f1ba bytes) 000266[0] 1778535085.427514 mapwrite 0x060b27 thru 0x06676c (0x5c46 bytes) 000267[1] 1778535085.543290 write 0x8f3188 thru 0x8f3b4a (0x9c3 bytes) 000268[1] 1778535085.556448 mapwrite 0x03869a thru 0x03fe2f (0x7796 bytes) 000270[0] 1778535085.685083 mapread 0x0c902a thru 0x0d38ee (0xa8c5 bytes) 000271[1] 1778535085.757513 write 0x4143cd thru 0x423b87 (0xf7bb bytes) 000273[0] 1778535085.771409 read 0x37012c thru 0x37bab2 (0xb987 bytes) 000274[1] 1778535085.797273 read 0x35d80c thru 0x36cec7 (0xf6bc bytes) 000275[0] 1778535085.861946 mapwrite 0x0a8d33 thru 0x0acf89 (0x4257 bytes) 000276[0] 1778535086.003258 trunc from 0x8f3b4b to 0x3b3e6e (0x53fcdc bytes) 000278[1] 1778535086.061423 write 0x1294d0 thru 0x12a028 (0xb59 bytes) 000279[1] 1778535086.091057 mapwrite 0x469480 thru 0x46d93e (0x44bf bytes) 000281[1] 1778535086.176651 mapread 0x031ce2 thru 0x03aff6 (0x9315 bytes) 000287[1] 1778535086.237575 trunc from 0x46d93f to 0x7aa832 (0x33cef4 bytes) 000288[1] 1778535086.263257 mapwrite 0x611dac thru 0x61ab97 (0x8dec bytes) 000289[0] 1778535086.372954 trunc from 0x7aa833 to 0x03c3d1 (0x76e461 bytes) 000291[1] 1778535086.431027 trunc from 0x03c3d2 to 0x107192 (0xcadc1 bytes) 000292[0] 1778535086.457977 read 0x08560c thru 0x08cbdb (0x75d0 bytes) 000295[1] 1778535086.547934 mapwrite 0x535a45 thru 0x542884 (0xce40 bytes) 000297[1] 1778535086.687654 trunc from 0x542885 to 0x51481c (0x2e068 bytes) 000301[1] 1778535086.720667 read 0x2acdac thru 0x2bb613 (0xe868 bytes) 000302[0] 1778535086.730588 trunc from 0x51481d to 0x11fe6f (0x3f49ad bytes) 000303[0] 1778535086.772181 trunc from 0x11fe70 to 0x1fb5c2 (0xdb753 bytes) 000304[1] 1778535086.796333 trunc from 0x1fb5c3 to 0x2238f0 (0x2832e bytes) 000305[1] 1778535086.831116 trunc from 0x2238f1 to 0x04e2b9 (0x1d5637 bytes) 000307[0] 1778535086.877865 mapread 0x03805f thru 0x043b44 (0xbae6 bytes) 000309[1] 1778535086.963431 trunc from 0x04e2ba to 0x78ca85 (0x73e7cc bytes) 000311[0] 1778535086.994453 trunc from 0x78ca86 to 0x2d0a8e (0x4bbff7 bytes) 000312[1] 1778535087.023740 mapread 0x2c9e19 thru 0x2d0a8e (0x6c76 bytes) 000313[0] 1778535087.069001 mapwrite 0x90cfb9 thru 0x918fe9 (0xc031 bytes) 000315[1] 1778535087.260193 mapread 0x7d4b5a thru 0x7dfd16 (0xb1bd bytes) 000318[0] 1778535087.350897 trunc from 0x918fea to 0x909f4f (0xf09a bytes) 000319[1] 1778535087.374030 trunc from 0x909f50 to 0x000479 (0x909ad6 bytes) 000320[0] 1778535087.417185 mapread 0x000366 thru 0x000479 (0x114 bytes) 000322[1] 1778535087.504164 write 0x49ef55 thru 0x49fa78 (0xb24 bytes) 000325[0] 1778535087.556524 trunc from 0x49fa79 to 0x5ec6e5 (0x14cc6d bytes) 000327[0] 1778535087.592160 mapread 0x2f4644 thru 0x2fb2ea (0x6ca7 bytes) 000328[1] 1778535087.620247 trunc from 0x5ec6e6 to 0x37003a (0x27c6ab bytes) 000329[0] 1778535087.660989 read 0x35a41e thru 0x36522e (0xae11 bytes) 000330[1] 1778535087.704347 mapread 0x211ddd thru 0x220df7 (0xf01b bytes) 000333[0] 1778535087.746566 trunc from 0x37003b to 0x669120 (0x2f90e6 bytes) 000337[0] 1778535087.842230 mapread 0x4ce389 thru 0x4d54d4 (0x714c bytes) 000340[1] 1778535087.894465 mapwrite 0x248304 thru 0x254c3c (0xc939 bytes) 000341[1] 1778535087.972320 mapwrite 0x95b439 thru 0x95b5a5 (0x16d bytes) 000342[1] 1778535088.050256 write 0x7e1a27 thru 0x7e2df0 (0x13ca bytes) 000343[0] 1778535088.067277 write 0x644c61 thru 0x652ea2 (0xe242 bytes) 000346[1] 1778535088.138881 write 0x84fb5b thru 0x850591 (0xa37 bytes) 000348[0] 1778535088.180138 mapread 0x6044ab thru 0x609605 (0x515b bytes) 000351[1] 1778535088.226515 mapread 0x3cb9b0 thru 0x3dad6a (0xf3bb bytes) 000352[1] 1778535088.275977 mapread 0x29f947 thru 0x2a4fb0 (0x566a bytes) 000353[1] 1778535088.309424 write 0x0543cd thru 0x05eca2 (0xa8d6 bytes) 000354[1] 1778535088.345212 read 0x1b48e4 thru 0x1c2a90 (0xe1ad bytes) 000355[1] 1778535088.438411 mapread 0x5a172d thru 0x5a8919 (0x71ed bytes) 000356[1] 1778535088.457568 mapread 0x81adb1 thru 0x8235c6 (0x8816 bytes) 000358[1] 1778535088.494334 mapwrite 0x7fac10 thru 0x804156 (0x9547 bytes) 000360[1] 1778535088.539044 read 0x6be1c0 thru 0x6c45d2 (0x6413 bytes) 000362[1] 1778535088.553236 mapread 0x92ce24 thru 0x92e06e (0x124b bytes) 000363[1] 1778535088.553927 mapwrite 0x591004 thru 0x595d6a (0x4d67 bytes) 000374[1] 1778535088.681417 mapread 0x266bff thru 0x26dbd7 (0x6fd9 bytes) 000376[0] 1778535088.713271 mapread 0x0b51ca thru 0x0c056d (0xb3a4 bytes) 000378[1] 1778535088.777635 mapwrite 0x4f4f9e thru 0x502153 (0xd1b6 bytes) 000379[0] 1778535088.898281 mapwrite 0x5b385a thru 0x5b4013 (0x7ba bytes) 000380[1] 1778535088.977890 mapread 0x04eb9a thru 0x0552e8 (0x674f bytes) 000381[0] 1778535089.035265 read 0x329067 thru 0x32e442 (0x53dc bytes) 000383[1] 1778535089.081619 trunc from 0x95b5a6 to 0x469529 (0x4f207c bytes) 000386[0] 1778535089.119993 write 0x1bcbbc thru 0x1c54aa (0x88ef bytes) 000390[1] 1778535089.187464 read 0x31479e thru 0x31c082 (0x78e5 bytes) 000392[0] 1778535089.236056 read 0x048d08 thru 0x050054 (0x734d bytes) 000395[1] 1778535089.295154 mapread 0x078bee thru 0x07c79f (0x3bb2 bytes) 000396[1] 1778535089.312970 mapwrite 0x8014b7 thru 0x806be5 (0x572f bytes) 000400[0] 1778535089.464889 mapwrite 0x91ba38 thru 0x9205bd (0x4b86 bytes) 000401[0] 1778535089.570685 mapwrite 0x861458 thru 0x86b904 (0xa4ad bytes) 000402[0] 1778535089.620525 mapwrite 0x7825ed thru 0x783fba (0x19ce bytes) 000404[1] 1778535089.663587 write 0x87400d thru 0x87ff37 (0xbf2b bytes) 000405[0] 1778535089.701084 mapwrite 0x870a1c thru 0x8727bd (0x1da2 bytes) 000407[0] 1778535089.794839 mapread 0x75f4f5 thru 0x767e32 (0x893e bytes) 000408[1] 1778535089.812730 write 0x0eff64 thru 0x0fa0ff (0xa19c bytes) 000410[0] 1778535089.858769 mapread 0x0a8fc4 thru 0x0b8d79 (0xfdb6 bytes) 000412[0] 1778535089.984070 mapwrite 0x4e568f thru 0x4f06cb (0xb03d bytes) 000414[1] 1778535090.108496 mapread 0x465bb0 thru 0x46da1f (0x7e70 bytes) 000415[0] 1778535090.191076 mapread 0x4c160e thru 0x4c54e4 (0x3ed7 bytes) 000417[1] 1778535090.264117 trunc from 0x9205be to 0x7157f9 (0x20adc4 bytes) 000418[0] 1778535090.307289 mapread 0x6a5900 thru 0x6b2ccf (0xd3d0 bytes) 000419[0] 1778535090.380463 read 0x04989c thru 0x05862b (0xed90 bytes) 000422[1] 1778535090.437632 mapwrite 0x7564e6 thru 0x761172 (0xac8d bytes) 000423[1] 1778535090.569080 write 0x088062 thru 0x08af4f (0x2eee bytes) 000425[0] 1778535090.580458 read 0x70744a thru 0x714559 (0xd110 bytes) 000426[1] 1778535090.603198 mapread 0x0ad660 thru 0x0b5867 (0x8208 bytes) 000427[1] 1778535090.654888 trunc from 0x761173 to 0x78e827 (0x2d6b5 bytes) 000428[1] 1778535090.691004 write 0x4624bf thru 0x469aaf (0x75f1 bytes) 000435[0] 1778535090.780869 mapwrite 0x0bb183 thru 0x0bf202 (0x4080 bytes) 000436[0] 1778535090.887911 trunc from 0x78e828 to 0x885eaa (0xf7683 bytes) 000437[1] 1778535090.917764 mapwrite 0x36046d thru 0x36092a (0x4be bytes) 000441[1] 1778535091.052404 mapread 0x120e11 thru 0x129a7b (0x8c6b bytes) 000446[0] 1778535091.185310 trunc from 0x885eab to 0x994143 (0x10e299 bytes) 000447[1] 1778535091.214868 mapread 0x59f8e9 thru 0x5a2fa4 (0x36bc bytes) 000451[0] 1778535091.285199 mapread 0x56de2a thru 0x577479 (0x9650 bytes) 000462[0] 1778535091.448775 write 0x3d6349 thru 0x3ded53 (0x8a0b bytes) 000466[0] 1778535091.505355 mapread 0x277ea3 thru 0x27a6fb (0x2859 bytes) 000467[1] 1778535091.525902 mapwrite 0x035723 thru 0x043b83 (0xe461 bytes) 000468[1] 1778535091.669449 trunc from 0x994144 to 0x226955 (0x76d7ee bytes) 000471[0] 1778535091.730687 mapread 0x1f01fb thru 0x1f51d2 (0x4fd8 bytes) 000472[0] 1778535091.771688 mapwrite 0x531110 thru 0x53f04f (0xdf40 bytes) 000474[1] 1778535091.940743 write 0x56005b thru 0x56fd49 (0xfcef bytes) 000475[1] 1778535091.971865 trunc from 0x56fd4a to 0x38da73 (0x1e22d6 bytes) 000476[0] 1778535092.026653 trunc from 0x38da74 to 0x5c993f (0x23becc bytes) 000477[0] 1778535092.064327 trunc from 0x5c9940 to 0x744142 (0x17a803 bytes) 000478[1] 1778535092.103323 read 0x2a1a64 thru 0x2a42b0 (0x284d bytes) 000479[0] 1778535092.140967 trunc from 0x744143 to 0x8d696d (0x19282b bytes) 000480[1] 1778535092.170547 read 0x5c70f9 thru 0x5cc30a (0x5212 bytes) 000484[0] 1778535092.251390 read 0x4999ee thru 0x4a33ec (0x99ff bytes) 000488[1] 1778535092.345491 trunc from 0x8d696e to 0x76be0d (0x16ab60 bytes) 000491[0] 1778535092.417906 mapread 0x3de8eb thru 0x3ea117 (0xb82d bytes) 000493[1] 1778535092.499051 trunc from 0x76be0e to 0x42c018 (0x33fdf5 bytes) 000494[0] 1778535092.514737 trunc from 0x42c019 to 0x713ba4 (0x2e7b8c bytes) 000496[1] 1778535092.595890 read 0x4a842e thru 0x4af106 (0x6cd9 bytes) 000498[1] 1778535092.619939 mapread 0x6109be thru 0x61c9e5 (0xc028 bytes) 000499[1] 1778535092.701227 trunc from 0x713ba5 to 0x10dc7b (0x605f29 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5092 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x872f63 000009[0] 1778535093.342272 trunc from 00000000 to 0x872f62 (0x872f63 bytes) 000010[0] 1778535093.370500 read_OD 0x295000 thru 0x295fff (0x1000 bytes) 000012[1] 1778535093.388250 mapwrite 0x530000 thru 0x53bf7c (0xbf7d bytes) 000014[1] 1778535093.536036 mapread 0x632000 thru 0x6394c2 (0x74c3 bytes) 000023[0] 1778535093.639245 mapread 0x6d8000 thru 0x6e069a (0x869b bytes) 000024[1] 1778535093.692694 write 0x3f8000 thru 0x3ffb97 (0x7b98 bytes) 000026[0] 1778535093.721190 mapread 0x783000 thru 0x78582a (0x282b bytes) 000028[1] 1778535093.760418 mapread 0x64b000 thru 0x64e84b (0x384c bytes) 000030[1] 1778535093.779090 read 0x6bb000 thru 0x6c1e42 (0x6e43 bytes) 000035[0] 1778535093.794139 trunc from 0x872f63 to 0x471150 (0x401e12 bytes) 000036[1] 1778535093.835001 mapread 0x21a000 thru 0x229fac (0xffad bytes) 000037[1] 1778535093.885523 mapread 0x415000 thru 0x41649e (0x149f bytes) 000038[1] 1778535093.893476 mapread 0x306000 thru 0x3127e0 (0xc7e1 bytes) 000040[0] 1778535093.919386 mapwrite 0x31d000 thru 0x32c30f (0xf310 bytes) 000041[1] 1778535094.008946 write 0x36c000 thru 0x377a0f (0xba10 bytes) 000042[1] 1778535094.021615 mapwrite 0x154000 thru 0x1580c0 (0x40c1 bytes) 000044[0] 1778535094.137515 trunc from 0x471151 to 0x21c768 (0x2549e8 bytes) 000045[1] 1778535094.163231 mapwrite 0x196000 thru 0x19c43e (0x643f bytes) 000048[0] 1778535094.259614 read_OD 0x006000 thru 0x014fff (0xf000 bytes) 000049[0] 1778535094.281136 mapwrite 0x921000 thru 0x924c11 (0x3c12 bytes) 000050[0] 1778535094.484709 mapwrite 0x41b000 thru 0x42aa3a (0xfa3b bytes) 000053[0] 1778535094.543402 mapwrite 0x466000 thru 0x475386 (0xf387 bytes) 000055[0] 1778535094.620508 write_OD 0x113000 thru 0x116fff (0x4000 bytes) 000056[0] 1778535094.679700 write_OD 0x909000 thru 0x913fff (0xb000 bytes) 000059[1] 1778535094.731518 read 0x258000 thru 0x25e4f0 (0x64f1 bytes) 000061[0] 1778535094.808597 mapwrite 0x80e000 thru 0x817096 (0x9097 bytes) 000062[0] 1778535094.936499 read_OD 0x7dd000 thru 0x7e7fff (0xb000 bytes) 000064[0] 1778535094.959274 mapread 0x10d000 thru 0x119b2a (0xcb2b bytes) 000066[0] 1778535095.034911 write_OD 0x80c000 thru 0x815fff (0xa000 bytes) 000067[1] 1778535095.092534 trunc from 0x924c12 to 0x4c32d2 (0x46193f bytes) 000069[0] 1778535095.144679 trunc from 0x4c32d3 to 0x34110e (0x1821c4 bytes) 000070[1] 1778535095.185556 mapread 0x30f000 thru 0x31cc04 (0xdc05 bytes) 000071[0] 1778535095.261557 mapwrite 0x830000 thru 0x839724 (0x9725 bytes) 000074[0] 1778535095.403896 trunc from 0x839725 to 0x5038d0 (0x335e54 bytes) 000075[0] 1778535095.427168 trunc from 0x5038d1 to 0x226d72 (0x2dcb5e bytes) 000076[0] 1778535095.446906 read_OD 0x044000 thru 0x050fff (0xd000 bytes) 000078[1] 1778535095.472651 write 0x231000 thru 0x231ed9 (0xeda bytes) 000079[0] 1778535095.501666 read_OD 0x075000 thru 0x081fff (0xd000 bytes) 000080[0] 1778535095.530157 trunc from 0x231eda to 0x4fbfb0 (0x2ca0d7 bytes) 000081[0] 1778535095.553619 trunc from 0x4fbfb1 to 0x48bf3e (0x70072 bytes) 000086[1] 1778535095.602491 mapwrite 0x59c000 thru 0x5a6a70 (0xaa71 bytes) 000088[0] 1778535095.738044 trunc from 0x5a6a71 to 0x647128 (0xa06b8 bytes) 000093[0] 1778535095.792115 read_OD 0x358000 thru 0x365fff (0xe000 bytes) 000094[1] 1778535095.803173 mapread 0x1f4000 thru 0x1fb117 (0x7118 bytes) 000095[1] 1778535095.822685 trunc from 0x647129 to 0x7efd0f (0x1a8be7 bytes) 000096[0] 1778535095.857776 write_OD 0x152000 thru 0x157fff (0x6000 bytes) truncating to largest ever: 0x9dd10f 000097[0] 1778535095.894019 trunc from 0x7efd10 to 0x9dd10e (0x1ed3ff bytes) 000100[0] 1778535095.929290 mapread 0x220000 thru 0x22d746 (0xd747 bytes) 000101[1] 1778535096.057713 mapread 0x0c1000 thru 0x0d000a (0xf00b bytes) 000104[1] 1778535096.120716 trunc from 0x9dd10f to 0x70bd1f (0x2d13ef bytes) 000105[0] 1778535096.160257 mapwrite 0x916000 thru 0x91a071 (0x4072 bytes) 000106[0] 1778535096.233552 mapread 0x0db000 thru 0x0ea2ff (0xf300 bytes) 000108[0] 1778535096.345560 trunc from 0x91a072 to 0x5f4fdd (0x325094 bytes) 000110[0] 1778535096.359517 read_OD 0x5a9000 thru 0x5a9fff (0x1000 bytes) 000111[0] 1778535096.371943 mapread 0x4f8000 thru 0x4fa8fb (0x28fc bytes) 000112[0] 1778535096.396228 mapwrite 0x356000 thru 0x363ff5 (0xdff6 bytes) 000113[0] 1778535096.512012 write_OD 0x6ce000 thru 0x6d6fff (0x9000 bytes) 000114[0] 1778535096.548399 mapwrite 0x622000 thru 0x624d44 (0x2d45 bytes) 000115[1] 1778535096.630351 read 0x4d4000 thru 0x4d9d37 (0x5d38 bytes) 000117[0] 1778535096.653295 mapread 0x623000 thru 0x62e69a (0xb69b bytes) 000118[0] 1778535096.725080 mapwrite 0x646000 thru 0x652481 (0xc482 bytes) 000119[1] 1778535096.847714 write 0x318000 thru 0x32198c (0x998d bytes) 000121[0] 1778535096.875596 write_OD 0x13c000 thru 0x149fff (0xe000 bytes) 000122[0] 1778535096.926309 trunc from 0x6d7000 to 0x2b40c4 (0x422f3b bytes) 000124[0] 1778535096.975539 mapread 0x0f9000 thru 0x10420c (0xb20d bytes) 000126[1] 1778535097.027137 mapread 0x25d000 thru 0x25e1b2 (0x11b3 bytes) 000127[0] 1778535097.053577 write_OD 0x849000 thru 0x853fff (0xb000 bytes) 000129[0] 1778535097.099478 trunc from 0x854000 to 0x967f83 (0x113f84 bytes) 000133[0] 1778535097.154874 mapread 0x26b000 thru 0x2723b4 (0x73b5 bytes) 000136[0] 1778535097.196658 write_OD 0x171000 thru 0x17ffff (0xf000 bytes) 000138[1] 1778535097.256171 trunc from 0x967f84 to 0x23ebdc (0x7293a7 bytes) 000141[0] 1778535097.278064 mapwrite 0x7e2000 thru 0x7e8db3 (0x6db4 bytes) 000142[1] 1778535097.429638 mapread 0x249000 thru 0x251faa (0x8fab bytes) 000143[1] 1778535097.462236 trunc from 0x7e8db4 to 0x2dbb30 (0x50d283 bytes) 000144[0] 1778535097.501494 trunc from 0x2dbb31 to 0x369c09 (0x8e0d9 bytes) 000145[1] 1778535097.517253 mapread 0x181000 thru 0x1836b6 (0x26b7 bytes) 000147[0] 1778535097.571594 mapwrite 0x65b000 thru 0x65ea61 (0x3a62 bytes) 000148[0] 1778535097.672396 mapread 0x48f000 thru 0x49d66d (0xe66e bytes) 000149[0] 1778535097.743569 read_OD 0x292000 thru 0x296fff (0x5000 bytes) 000150[1] 1778535097.761433 read 0x0e4000 thru 0x0eec0c (0xac0d bytes) 000151[1] 1778535097.814114 trunc from 0x65ea62 to 0x6024ea (0x5c577 bytes) 000152[0] 1778535097.865246 trunc from 0x6024eb to 0x541de1 (0xc0709 bytes) 000155[0] 1778535097.946766 trunc from 0x541de2 to 0x91d310 (0x3db52f bytes) 000156[0] 1778535097.988573 mapread 0x8b8000 thru 0x8ba2a6 (0x22a7 bytes) 000158[0] 1778535098.055322 mapread 0x739000 thru 0x7408bb (0x78bc bytes) 000159[1] 1778535098.055844 mapread 0x6b8000 thru 0x6bf292 (0x7293 bytes) 000161[0] 1778535098.110999 mapwrite 0x979000 thru 0x980d38 (0x7d39 bytes) 000162[0] 1778535098.239493 mapread 0x6f6000 thru 0x705863 (0xf864 bytes) 000163[1] 1778535098.319195 write 0x8d0000 thru 0x8d0374 (0x375 bytes) 000164[0] 1778535098.335645 read_OD 0x74f000 thru 0x75bfff (0xd000 bytes) 000165[1] 1778535098.366506 mapwrite 0x56e000 thru 0x572072 (0x4073 bytes) 000166[0] 1778535098.470944 mapread 0x325000 thru 0x3278e2 (0x28e3 bytes) 000168[0] 1778535098.561362 write_OD 0x123000 thru 0x12cfff (0xa000 bytes) 000171[1] 1778535098.612966 mapread 0x565000 thru 0x5688fc (0x38fd bytes) 000172[1] 1778535098.634065 write 0x142000 thru 0x142b76 (0xb77 bytes) 000173[1] 1778535098.666646 read 0x319000 thru 0x31e17c (0x517d bytes) 000175[0] 1778535098.753473 mapwrite 0x7f1000 thru 0x7ff6a6 (0xe6a7 bytes) 000180[0] 1778535098.924220 read_OD 0x2ca000 thru 0x2cefff (0x5000 bytes) 000181[1] 1778535098.936380 write 0x186000 thru 0x193976 (0xd977 bytes) 000184[0] 1778535099.006058 trunc from 0x980d39 to 0x592deb (0x3edf4d bytes) 000187[1] 1778535099.056620 write 0x2c0000 thru 0x2cb2e6 (0xb2e7 bytes) 000188[1] 1778535099.098934 read 0x069000 thru 0x07712b (0xe12c bytes) 000189[1] 1778535099.184921 trunc from 0x592dec to 0x043f2d (0x54eebe bytes) 000193[1] 1778535099.236648 trunc from 0x043f2e to 0x87cdf3 (0x838ec6 bytes) 000195[1] 1778535099.273047 mapwrite 0x493000 thru 0x49b400 (0x8401 bytes) 000199[1] 1778535099.469906 read 0x136000 thru 0x13e3b2 (0x83b3 bytes) 000200[0] 1778535099.522257 write_OD 0x64e000 thru 0x64ffff (0x2000 bytes) 000208[0] 1778535099.585652 mapread 0x706000 thru 0x707ff7 (0x1ff8 bytes) 000209[0] 1778535099.661384 mapread 0x001000 thru 0x004a1b (0x3a1c bytes) 000210[1] 1778535099.692838 write 0x06a000 thru 0x07259b (0x859c bytes) 000212[1] 1778535099.729198 mapwrite 0x293000 thru 0x29956c (0x656d bytes) 000214[1] 1778535099.820219 read 0x614000 thru 0x61b502 (0x7503 bytes) 000216[0] 1778535099.881979 read_OD 0x69a000 thru 0x6a2fff (0x9000 bytes) 000217[1] 1778535099.914151 trunc from 0x87cdf4 to 0x8a3c06 (0x26e13 bytes) 000219[1] 1778535099.947583 write 0x73f000 thru 0x7400f6 (0x10f7 bytes) 000223[0] 1778535099.982763 trunc from 0x8a3c07 to 0x596f1b (0x30cceb bytes) 000224[1] 1778535100.007434 write 0x2f2000 thru 0x2f3120 (0x1121 bytes) 000225[1] 1778535100.030980 mapread 0x54e000 thru 0x551d06 (0x3d07 bytes) 000226[1] 1778535100.053270 write 0x8af000 thru 0x8b7f2d (0x8f2e bytes) 000228[1] 1778535100.058489 trunc from 0x8b7f2e to 0x6fa013 (0x1bdf1a bytes) 000229[0] 1778535100.084431 write_OD 0x3cb000 thru 0x3d1fff (0x7000 bytes) 000232[0] 1778535100.142396 trunc from 0x6fa014 to 0x1ccd34 (0x52d2df bytes) 000237[0] 1778535100.209578 mapread 0x0c4000 thru 0x0cb307 (0x7308 bytes) 000239[1] 1778535100.253461 read 0x0ee000 thru 0x0f19a0 (0x39a1 bytes) 000241[0] 1778535100.285051 mapwrite 0x4a4000 thru 0x4a85ca (0x45cb bytes) 000243[0] 1778535100.485563 mapwrite 0x0d0000 thru 0x0d63ba (0x63bb bytes) 000244[1] 1778535100.596939 read 0x3dd000 thru 0x3e62a2 (0x92a3 bytes) 000246[1] 1778535100.634043 mapwrite 0x6a6000 thru 0x6b5c11 (0xfc12 bytes) 000248[1] 1778535100.777706 read 0x145000 thru 0x1512e5 (0xc2e6 bytes) 000249[0] 1778535100.841690 read_OD 0x672000 thru 0x67efff (0xd000 bytes) 000250[1] 1778535100.884313 trunc from 0x6b5c12 to 0x98eb4f (0x2d8f3e bytes) 000251[1] 1778535100.932097 write 0x9aa000 thru 0x9aff9d (0x5f9e bytes) 000253[1] 1778535100.949453 mapread 0x172000 thru 0x17dd09 (0xbd0a bytes) 000254[1] 1778535101.065999 trunc from 0x9aff9e to 0x656704 (0x359899 bytes) 000256[1] 1778535101.127843 mapwrite 0x968000 thru 0x969e41 (0x1e42 bytes) 000259[0] 1778535101.197770 write_OD 0x535000 thru 0x53efff (0xa000 bytes) 000261[0] 1778535101.249574 mapwrite 0x0e3000 thru 0x0e8292 (0x5293 bytes) 000263[0] 1778535101.409511 mapwrite 0x5b6000 thru 0x5be9e6 (0x89e7 bytes) 000265[1] 1778535101.533668 mapwrite 0x0bf000 thru 0x0c7d22 (0x8d23 bytes) 000267[1] 1778535101.645264 mapwrite 0x340000 thru 0x349b51 (0x9b52 bytes) 000269[0] 1778535101.779519 mapwrite 0x109000 thru 0x10d036 (0x4037 bytes) 000270[1] 1778535101.880251 mapwrite 0x12b000 thru 0x138a73 (0xda74 bytes) 000274[0] 1778535102.066883 write_OD 0x6d1000 thru 0x6d2fff (0x2000 bytes) 000275[0] 1778535102.117849 write_OD 0x142000 thru 0x14bfff (0xa000 bytes) 000277[1] 1778535102.168643 mapread 0x851000 thru 0x85bac4 (0xaac5 bytes) 000280[0] 1778535102.239708 mapread 0x0b8000 thru 0x0bdf6f (0x5f70 bytes) 000284[1] 1778535102.285923 mapwrite 0x3cc000 thru 0x3d5ea4 (0x9ea5 bytes) 000286[0] 1778535102.408562 mapread 0x36f000 thru 0x37a77f (0xb780 bytes) 000287[1] 1778535102.520711 mapread 0x7ea000 thru 0x7f7869 (0xd86a bytes) 000289[0] 1778535102.639627 write_OD 0x4fd000 thru 0x506fff (0xa000 bytes) 000296[1] 1778535102.698588 mapwrite 0x6f1000 thru 0x6f2f79 (0x1f7a bytes) 000299[1] 1778535102.803146 trunc from 0x969e42 to 0x0dba08 (0x88e439 bytes) 000300[1] 1778535102.853581 trunc from 0x0dba09 to 0x13150c (0x55b04 bytes) 000302[0] 1778535102.929392 read_OD 0x04e000 thru 0x054fff (0x7000 bytes) 000303[0] 1778535102.965378 mapwrite 0x718000 thru 0x71fcc5 (0x7cc6 bytes) 000304[1] 1778535103.152904 trunc from 0x71fcc6 to 0x12a36f (0x5f5956 bytes) 000309[1] 1778535103.212425 mapwrite 0x399000 thru 0x39edd4 (0x5dd5 bytes) 000311[1] 1778535103.348697 mapread 0x39d000 thru 0x39e68e (0x168f bytes) 000313[0] 1778535103.390685 read_OD 0x19a000 thru 0x1a4fff (0xb000 bytes) 000314[1] 1778535103.424066 write 0x403000 thru 0x4051f4 (0x21f5 bytes) 000316[1] 1778535103.448760 mapread 0x05f000 thru 0x06b589 (0xc58a bytes) 000317[1] 1778535103.499421 trunc from 0x4051f5 to 0x101826 (0x3039ce bytes) 000330[1] 1778535103.621411 mapread 0x07a000 thru 0x07f1b0 (0x51b1 bytes) 000333[1] 1778535103.663594 mapread 0x0bc000 thru 0x0c46b2 (0x86b3 bytes) 000334[1] 1778535103.731281 mapwrite 0x65b000 thru 0x663f07 (0x8f08 bytes) 000337[1] 1778535103.915258 mapwrite 0x529000 thru 0x52b018 (0x2019 bytes) 000339[0] 1778535104.010355 trunc from 0x663f08 to 0x1e3590 (0x480977 bytes) 000341[0] 1778535104.037650 trunc from 0x1e3591 to 0x7422d1 (0x55ed41 bytes) 000342[1] 1778535104.056826 mapread 0x25d000 thru 0x26007f (0x3080 bytes) 000345[1] 1778535104.110338 mapread 0x44f000 thru 0x458f86 (0x9f87 bytes) 000346[1] 1778535104.152023 trunc from 0x7422d2 to 0x4e3013 (0x25f2be bytes) 000349[0] 1778535104.184689 mapread 0x44c000 thru 0x45100d (0x500e bytes) 000352[0] 1778535104.306420 write_OD 0x23a000 thru 0x23afff (0x1000 bytes) 000353[0] 1778535104.366963 write_OD 0x751000 thru 0x75bfff (0xb000 bytes) 000355[0] 1778535104.429063 mapread 0x644000 thru 0x650a4d (0xca4e bytes) 000356[0] 1778535104.506769 mapwrite 0x85d000 thru 0x86311f (0x6120 bytes) 000357[0] 1778535104.659397 trunc from 0x863120 to 0x477788 (0x3eb997 bytes) 000358[1] 1778535104.679618 mapwrite 0x353000 thru 0x358a97 (0x5a98 bytes) 000359[1] 1778535104.806760 trunc from 0x477789 to 0x7abac6 (0x33433e bytes) 000360[0] 1778535104.827339 mapwrite 0x0bf000 thru 0x0c65ba (0x75bb bytes) 000362[1] 1778535104.975983 mapwrite 0x466000 thru 0x471758 (0xb759 bytes) 000363[1] 1778535105.130297 write 0x51b000 thru 0x52a9ac (0xf9ad bytes) 000366[0] 1778535105.174775 read_OD 0x43c000 thru 0x444fff (0x9000 bytes) 000368[1] 1778535105.225752 trunc from 0x7abac7 to 0x4f9f33 (0x2b1b93 bytes) 000369[0] 1778535105.294299 mapwrite 0x400000 thru 0x40e6d4 (0xe6d5 bytes) 000370[0] 1778535105.475012 read_OD 0x39e000 thru 0x3a4fff (0x7000 bytes) 000372[1] 1778535105.496868 mapread 0x0a0000 thru 0x0a466e (0x466f bytes) 000373[0] 1778535105.520108 mapwrite 0x0d0000 thru 0x0d05e3 (0x5e4 bytes) 000374[0] 1778535105.597914 write_OD 0x512000 thru 0x516fff (0x5000 bytes) 000375[0] 1778535105.647301 read_OD 0x3ac000 thru 0x3b2fff (0x7000 bytes) 000378[1] 1778535105.659421 mapwrite 0x4f5000 thru 0x502c73 (0xdc74 bytes) 000379[0] 1778535105.787247 mapread 0x03a000 thru 0x03ef4e (0x4f4f bytes) 000380[1] 1778535105.852157 read 0x076000 thru 0x07c119 (0x611a bytes) 000385[0] 1778535105.904854 read_OD 0x305000 thru 0x306fff (0x2000 bytes) 000386[1] 1778535105.917135 mapwrite 0x196000 thru 0x1a075a (0xa75b bytes) 000387[0] 1778535106.094775 mapwrite 0x71e000 thru 0x72b34c (0xd34d bytes) 000388[0] 1778535106.247537 mapwrite 0x3d5000 thru 0x3d985b (0x485c bytes) 000389[1] 1778535106.314546 read 0x251000 thru 0x251af6 (0xaf7 bytes) 000390[0] 1778535106.421195 write_OD 0x9aa000 thru 0x9b3fff (0xa000 bytes) 000393[1] 1778535106.489733 mapread 0x1d2000 thru 0x1e1723 (0xf724 bytes) 000394[0] 1778535106.625026 mapwrite 0x76b000 thru 0x776dec (0xbded bytes) 000395[0] 1778535106.801737 write_OD 0x112000 thru 0x116fff (0x5000 bytes) 000398[1] 1778535106.847210 write 0x20c000 thru 0x20ebbb (0x2bbc bytes) 000405[1] 1778535106.911110 mapwrite 0x234000 thru 0x23b757 (0x7758 bytes) 000407[0] 1778535107.016549 read_OD 0x658000 thru 0x65efff (0x7000 bytes) 000409[0] 1778535107.040823 write_OD 0x764000 thru 0x76afff (0x7000 bytes) 000410[1] 1778535107.106466 read 0x081000 thru 0x08b66d (0xa66e bytes) 000411[0] 1778535107.201227 mapwrite 0x5d5000 thru 0x5df048 (0xa049 bytes) 000413[1] 1778535107.364035 mapread 0x7ab000 thru 0x7b94fd (0xe4fe bytes) 000416[0] 1778535107.454310 mapread 0x24c000 thru 0x255267 (0x9268 bytes) 000418[0] 1778535107.522275 mapread 0x41f000 thru 0x41fbda (0xbdb bytes) 000420[1] 1778535107.528649 mapwrite 0x234000 thru 0x236534 (0x2535 bytes) 000421[1] 1778535107.586211 read 0x431000 thru 0x43f613 (0xe614 bytes) 000422[1] 1778535107.649440 read 0x5d4000 thru 0x5d65df (0x25e0 bytes) 000426[1] 1778535107.699116 mapwrite 0x5b6000 thru 0x5be008 (0x8009 bytes) 000429[1] 1778535107.756556 mapread 0x61b000 thru 0x624b6e (0x9b6f bytes) 000430[0] 1778535107.757160 mapwrite 0x5fa000 thru 0x5fcf2b (0x2f2c bytes) 000432[0] 1778535107.847843 mapwrite 0x404000 thru 0x40ba24 (0x7a25 bytes) 000434[1] 1778535107.935636 mapread 0x0bd000 thru 0x0c64d8 (0x94d9 bytes) 000437[0] 1778535108.014485 trunc from 0x9b4000 to 0x52149d (0x492b62 bytes) 000439[1] 1778535108.069336 write 0x652000 thru 0x654191 (0x2192 bytes) 000440[0] 1778535108.093054 write_OD 0x440000 thru 0x44dfff (0xe000 bytes) 000442[1] 1778535108.154533 mapread 0x1c4000 thru 0x1ca5a7 (0x65a8 bytes) 000443[1] 1778535108.184279 write 0x8f4000 thru 0x900ced (0xccee bytes) 000445[1] 1778535108.207070 write 0x275000 thru 0x27fc73 (0xac74 bytes) 000448[1] 1778535108.231136 mapwrite 0x578000 thru 0x57b515 (0x3516 bytes) 000450[0] 1778535108.340677 mapwrite 0x8a6000 thru 0x8afc77 (0x9c78 bytes) 000452[0] 1778535108.441060 trunc from 0x900cee to 0x8128f2 (0xee3fb bytes) 000453[1] 1778535108.474986 mapread 0x473000 thru 0x473a72 (0xa73 bytes) 000455[1] 1778535108.499513 mapread 0x417000 thru 0x41daad (0x6aae bytes) 000456[0] 1778535108.548479 mapread 0x579000 thru 0x587d17 (0xed18 bytes) 000457[1] 1778535108.615625 write 0x606000 thru 0x60afeb (0x4fec bytes) 000459[0] 1778535108.669766 mapwrite 0x126000 thru 0x13337d (0xd37e bytes) 000461[1] 1778535108.820304 read 0x11f000 thru 0x122d98 (0x3d99 bytes) 000463[0] 1778535108.893217 mapread 0x267000 thru 0x275576 (0xe577 bytes) 000464[0] 1778535108.989554 mapread 0x10f000 thru 0x113135 (0x4136 bytes) 000468[1] 1778535109.012172 trunc from 0x8128f3 to 0x953ded (0x1414fb bytes) 000470[0] 1778535109.071656 write_OD 0x0d7000 thru 0x0e3fff (0xd000 bytes) 000471[0] 1778535109.117125 read_OD 0x39a000 thru 0x3a1fff (0x8000 bytes) 000472[1] 1778535109.127436 mapwrite 0x9c8000 thru 0x9c9b28 (0x1b29 bytes) 000473[1] 1778535109.226842 write 0x134000 thru 0x142e85 (0xee86 bytes) 000474[0] 1778535109.256097 mapwrite 0x13a000 thru 0x146c76 (0xcc77 bytes) 000477[1] 1778535109.536175 trunc from 0x9c9b29 to 0x978a08 (0x51120 bytes) 000478[0] 1778535109.601641 mapread 0x0a6000 thru 0x0af3b0 (0x93b1 bytes) 000479[1] 1778535109.689622 mapwrite 0x454000 thru 0x45e96a (0xa96b bytes) 000480[1] 1778535109.817214 read 0x229000 thru 0x22bacb (0x2acc bytes) 000483[1] 1778535109.894577 write 0x836000 thru 0x8360f9 (0xfa bytes) 000484[1] 1778535109.912235 trunc from 0x978a09 to 0x69e336 (0x2da6d2 bytes) 000490[1] 1778535109.988055 write 0x1a5000 thru 0x1a8b17 (0x3b18 bytes) 000498[1] 1778535110.051255 mapwrite 0x95f000 thru 0x96b18e (0xc18f bytes) 000499[1] 1778535110.181736 write 0x798000 thru 0x79a89a (0x289b bytes) 000500[0] 1778535110.185629 write_OD 0x74e000 thru 0x751fff (0x4000 bytes) All operations completed A-OK! PASS 16a (39s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 17:31:53 (1778535113) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.21374 s, 9.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.137474 s, 15.3 MB/s Chance of close/open is 1 in 50 Seed set to 5114 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000003[0] 1778535114.980826 mapwrite 0x40fe32 thru 0x410d77 (0xf46 bytes) 000004[0] 1778535115.104321 mapread 0x3e5361 thru 0x3e646a (0x110a bytes) 000006[1] 1778535115.135405 mapwrite 0x0272be thru 0x028b86 (0x18c9 bytes) truncating to largest ever: 0x6326d3 000008[0] 1778535115.238244 trunc from 0x410d78 to 0x6326d2 (0x22195b bytes) truncating to largest ever: 0x88305f 000012[0] 1778535115.339588 trunc from 0x6326d3 to 0x88305e (0x25098c bytes) 000013[0] 1778535115.381977 mapwrite 0x6f48f7 thru 0x6f4e77 (0x581 bytes) 000014[1] 1778535115.421124 read 0x834883 thru 0x8348d9 (0x57 bytes) 000015[1] 1778535115.446025 mapwrite 0x69a195 thru 0x69ac4e (0xaba bytes) 000016[1] 1778535115.506268 mapread 0x4f0650 thru 0x4f1f5a (0x190b bytes) 000017[0] 1778535115.533176 mapwrite 0x3059f6 thru 0x30607c (0x687 bytes) 000018[1] 1778535115.600396 trunc from 0x88305f to 0x58910f (0x2f9f4f bytes) 000022[1] 1778535115.668749 read 0x426f34 thru 0x427236 (0x303 bytes) 000023[0] 1778535115.690709 trunc from 0x589110 to 0x80c901 (0x2837f2 bytes) 000026[1] 1778535115.777814 read 0x5cadb3 thru 0x5cbab1 (0xcff bytes) 000028[0] 1778535115.823603 mapwrite 0x3b11d4 thru 0x3b152c (0x359 bytes) 000029[0] 1778535115.907395 mapwrite 0x01cd5d thru 0x01d5ed (0x891 bytes) 000030[1] 1778535115.967034 trunc from 0x80c902 to 0x58c587 (0x28037a bytes) 000031[0] 1778535116.022255 mapread 0x1c309a thru 0x1c48d0 (0x1837 bytes) 000032[1] 1778535116.067194 write 0x436bc4 thru 0x437b8a (0xfc7 bytes) 000033[1] 1778535116.076351 write 0x320d07 thru 0x32265a (0x1954 bytes) 000034[1] 1778535116.097180 trunc from 0x58c588 to 0x6c5db9 (0x139832 bytes) 000036[1] 1778535116.115761 mapread 0x3844e5 thru 0x384967 (0x483 bytes) 000037[0] 1778535116.148575 mapwrite 0x9823b8 thru 0x9833eb (0x1034 bytes) 000039[1] 1778535116.261074 mapread 0x137c0d thru 0x138b1d (0xf11 bytes) 000041[0] 1778535116.304076 write 0x3e1447 thru 0x3e2c43 (0x17fd bytes) 000044[0] 1778535116.339735 write 0x33a737 thru 0x33afe0 (0x8aa bytes) 000045[0] 1778535116.356822 trunc from 0x9833ec to 0x3f8572 (0x58ae79 bytes) 000047[0] 1778535116.383296 mapwrite 0x54c38d thru 0x54c5c3 (0x237 bytes) truncating to largest ever: 0x8a5b70 000049[0] 1778535116.484819 trunc from 0x54c5c4 to 0x8a5b6f (0x3595ac bytes) 000050[1] 1778535116.528598 read 0x68a617 thru 0x68ba65 (0x144f bytes) 000052[1] 1778535116.561273 mapwrite 0x49cdc0 thru 0x49e6d3 (0x1914 bytes) 000053[1] 1778535116.633350 write 0x83af8c thru 0x83cde8 (0x1e5d bytes) 000055[0] 1778535116.681039 read 0x517a50 thru 0x5192fe (0x18af bytes) 000056[1] 1778535116.705538 write 0x3a60d8 thru 0x3a7869 (0x1792 bytes) 000057[0] 1778535116.746463 read 0x684b14 thru 0x6869f5 (0x1ee2 bytes) 000059[1] 1778535116.794451 read 0x2ca0dc thru 0x2cb28d (0x11b2 bytes) 000061[0] 1778535116.879255 mapread 0x49dc83 thru 0x49e31a (0x698 bytes) 000068[1] 1778535116.946186 read 0x6347a9 thru 0x634834 (0x8c bytes) 000069[0] 1778535116.964598 read 0x30c6dd thru 0x30c87f (0x1a3 bytes) 000071[1] 1778535117.078291 trunc from 0x8a5b70 to 0x5b7123 (0x2eea4c bytes) 000074[1] 1778535117.173131 mapread 0x0b2202 thru 0x0b3ebe (0x1cbd bytes) 000077[0] 1778535117.200078 mapwrite 0x8a956a thru 0x8ab04b (0x1ae2 bytes) 000079[1] 1778535117.349419 mapread 0x18559b thru 0x186966 (0x13cc bytes) 000081[0] 1778535117.415423 mapread 0x11bf7c thru 0x11d936 (0x19bb bytes) 000084[1] 1778535117.433349 read 0x519942 thru 0x51b326 (0x19e5 bytes) 000085[1] 1778535117.472975 mapwrite 0x3a7e1e thru 0x3a9b43 (0x1d26 bytes) 000088[0] 1778535117.598886 write 0x7084aa thru 0x709196 (0xced bytes) 000089[0] 1778535117.637104 mapread 0x393f21 thru 0x3945db (0x6bb bytes) 000091[1] 1778535117.647543 write 0x621141 thru 0x62265d (0x151d bytes) 000094[1] 1778535117.729265 mapread 0x870499 thru 0x870760 (0x2c8 bytes) 000101[1] 1778535117.835487 trunc from 0x8ab04c to 0x0923ec (0x818c5f bytes) 000102[0] 1778535117.875545 mapwrite 0x1e4d44 thru 0x1e51d2 (0x48f bytes) 000104[0] 1778535117.989020 trunc from 0x1e51d3 to 0x7da6a1 (0x5f54cf bytes) 000106[1] 1778535118.009538 read 0x7a171f thru 0x7a2187 (0xa69 bytes) 000109[0] 1778535118.050286 mapread 0x737903 thru 0x738dd7 (0x14d5 bytes) 000110[0] 1778535118.057901 write 0x16d0c8 thru 0x16d5c9 (0x502 bytes) 000111[1] 1778535118.087331 mapread 0x511703 thru 0x51367b (0x1f79 bytes) 000112[0] 1778535118.134709 mapwrite 0x2a588f thru 0x2a6241 (0x9b3 bytes) 000115[1] 1778535118.198116 mapwrite 0x7c3a2d thru 0x7c4203 (0x7d7 bytes) 000116[0] 1778535118.260533 mapwrite 0x2a7502 thru 0x2a89a7 (0x14a6 bytes) 000117[0] 1778535118.338648 mapread 0x03391a thru 0x034f8a (0x1671 bytes) 000118[1] 1778535118.380857 write 0x8a614d thru 0x8a7c22 (0x1ad6 bytes) 000119[1] 1778535118.397707 mapwrite 0x389c59 thru 0x38a651 (0x9f9 bytes) 000121[0] 1778535118.496972 trunc from 0x8a7c23 to 0x0e12d6 (0x7c694c bytes) 000123[1] 1778535118.551578 mapread 0x00c1bd thru 0x00cb6c (0x9b0 bytes) truncating to largest ever: 0x9b016b 000124[0] 1778535118.579470 trunc from 0x0e12d7 to 0x9b016a (0x8cee94 bytes) 000126[1] 1778535118.620707 mapwrite 0x2df6c3 thru 0x2e0cfe (0x163c bytes) 000127[0] 1778535118.723144 mapwrite 0x7ab3b7 thru 0x7abc7e (0x8c8 bytes) 000133[0] 1778535118.883866 mapread 0x10799b thru 0x107ed4 (0x53a bytes) 000134[1] 1778535118.920280 mapread 0x69b211 thru 0x69c5b6 (0x13a6 bytes) 000138[0] 1778535118.986863 write 0x8842cb thru 0x885688 (0x13be bytes) 000139[0] 1778535119.017509 mapwrite 0x5741d5 thru 0x574385 (0x1b1 bytes) 000141[1] 1778535119.069475 write 0x999372 thru 0x999709 (0x398 bytes) 000142[1] 1778535119.107595 mapwrite 0x0f4504 thru 0x0f5e8c (0x1989 bytes) 000146[0] 1778535119.191054 mapwrite 0x90bba4 thru 0x90c652 (0xaaf bytes) 000147[1] 1778535119.288239 trunc from 0x9b016b to 0x14785f (0x86890b bytes) 000148[1] 1778535119.314901 trunc from 0x147860 to 0x5a0887 (0x459028 bytes) 000152[0] 1778535119.355748 write 0x7549e9 thru 0x755fa8 (0x15c0 bytes) 000153[0] 1778535119.382079 write 0x65d35a thru 0x65d97a (0x621 bytes) 000154[1] 1778535119.423421 read 0x601cc3 thru 0x601cda (0x18 bytes) 000155[0] 1778535119.460872 write 0x4df003 thru 0x4df199 (0x197 bytes) 000156[1] 1778535119.476159 write 0x8f8fb1 thru 0x8faebd (0x1f0d bytes) 000158[0] 1778535119.564016 write 0x61682b thru 0x618660 (0x1e36 bytes) 000159[1] 1778535119.585763 read 0x5f5694 thru 0x5f6606 (0xf73 bytes) 000160[0] 1778535119.603196 trunc from 0x8faebe to 0x850435 (0xaaa88 bytes) 000163[0] 1778535119.639953 read 0x848ecb thru 0x849650 (0x786 bytes) 000164[0] 1778535119.643907 write 0x26bc1f thru 0x26c2d2 (0x6b4 bytes) 000165[0] 1778535119.673099 read 0x6d0c2a thru 0x6d2c15 (0x1fec bytes) 000166[1] 1778535119.696241 mapread 0x8107a3 thru 0x8121e8 (0x1a46 bytes) 000169[1] 1778535119.757547 write 0x116377 thru 0x117056 (0xce0 bytes) 000170[1] 1778535119.784877 read 0x48f623 thru 0x490532 (0xf10 bytes) 000171[1] 1778535119.852602 write 0x80ff67 thru 0x8119e6 (0x1a80 bytes) 000173[1] 1778535119.900621 read 0x165645 thru 0x165bdc (0x598 bytes) 000174[0] 1778535119.970010 write 0x36546c thru 0x3665bc (0x1151 bytes) 000175[1] 1778535120.021847 trunc from 0x850436 to 0x98a8b9 (0x13a484 bytes) 000178[0] 1778535120.074104 write 0x18ed0c thru 0x18ff63 (0x1258 bytes) 000179[0] 1778535120.102000 read 0x5807e0 thru 0x581720 (0xf41 bytes) 000180[1] 1778535120.134183 trunc from 0x98a8ba to 0x2376f8 (0x7531c1 bytes) 000182[0] 1778535120.199252 trunc from 0x2376f9 to 0x636d1a (0x3ff622 bytes) 000184[0] 1778535120.267664 read 0x0b2275 thru 0x0b32bf (0x104b bytes) 000186[0] 1778535120.338360 read 0x3a3f6e thru 0x3a505b (0x10ee bytes) 000187[0] 1778535120.361393 read 0x5cbf67 thru 0x5cc927 (0x9c1 bytes) 000190[1] 1778535120.379823 read 0x5425ec thru 0x54427f (0x1c94 bytes) 000192[0] 1778535120.430105 trunc from 0x636d1b to 0x068194 (0x5ceb86 bytes) 000193[1] 1778535120.477077 mapwrite 0x3dc2ca thru 0x3dc395 (0xcc bytes) 000194[1] 1778535120.579340 trunc from 0x3dc396 to 0x82e6ee (0x452359 bytes) 000198[0] 1778535120.679625 write 0x6da848 thru 0x6dc130 (0x18e9 bytes) 000199[1] 1778535120.711110 trunc from 0x82e6ef to 0x6ed639 (0x1410b5 bytes) 000201[0] 1778535120.773874 trunc from 0x6ed63a to 0x47cb03 (0x270b36 bytes) 000202[1] 1778535120.826224 write 0x5fa452 thru 0x5fa906 (0x4b5 bytes) 000204[1] 1778535120.885093 write 0x05ca09 thru 0x05d9b1 (0xfa9 bytes) 000205[1] 1778535120.912983 read 0x109fa4 thru 0x10ae0d (0xe6a bytes) 000208[1] 1778535121.007454 write 0x884d87 thru 0x885477 (0x6f1 bytes) 000212[1] 1778535121.010648 write 0x7d3bbf thru 0x7d542f (0x1871 bytes) 000213[0] 1778535121.045734 write 0x1c37df thru 0x1c419f (0x9c1 bytes) 000214[1] 1778535121.113607 mapread 0x3e6910 thru 0x3e79c9 (0x10ba bytes) 000216[0] 1778535121.157760 mapwrite 0x8d0846 thru 0x8d0ceb (0x4a6 bytes) 000219[1] 1778535121.269166 write 0x783a8e thru 0x785972 (0x1ee5 bytes) 000220[0] 1778535121.304296 write 0x0cae6d thru 0x0cc358 (0x14ec bytes) 000221[0] 1778535121.330060 trunc from 0x8d0cec to 0x96bf58 (0x9b26d bytes) 000224[0] 1778535121.431760 trunc from 0x96bf59 to 0x11dc21 (0x84e337 bytes) 000225[0] 1778535121.453718 mapwrite 0x768545 thru 0x7690b1 (0xb6d bytes) 000226[0] 1778535121.549407 write 0x65c5cc thru 0x65d971 (0x13a6 bytes) 000227[1] 1778535121.592386 read 0x1b67ea thru 0x1b8223 (0x1a3a bytes) 000228[1] 1778535121.634100 trunc from 0x7690b2 to 0x8fba0a (0x192959 bytes) 000229[1] 1778535121.686196 mapwrite 0x3fe86f thru 0x3ff5c8 (0xd5a bytes) 000231[0] 1778535121.857052 mapread 0x57bf01 thru 0x57c0ee (0x1ee bytes) 000234[1] 1778535121.903484 mapread 0x71d00f thru 0x71dd60 (0xd52 bytes) 000238[0] 1778535121.917719 mapread 0x66455f thru 0x665c45 (0x16e7 bytes) 000239[0] 1778535121.969982 read 0x656d66 thru 0x657b87 (0xe22 bytes) 000240[1] 1778535122.012888 mapwrite 0x522192 thru 0x5226a4 (0x513 bytes) 000242[0] 1778535122.098575 read 0x331f2b thru 0x333bbc (0x1c92 bytes) 000244[0] 1778535122.175763 read 0x074fb0 thru 0x0755ff (0x650 bytes) 000246[1] 1778535122.232580 mapread 0x3895be thru 0x389bcd (0x610 bytes) 000248[1] 1778535122.240584 write 0x255431 thru 0x256783 (0x1353 bytes) 000249[0] 1778535122.289427 mapread 0x39a133 thru 0x39b58c (0x145a bytes) 000250[1] 1778535122.314656 mapwrite 0x9d7640 thru 0x9d8842 (0x1203 bytes) 000251[1] 1778535122.431890 mapwrite 0x7770b9 thru 0x7771da (0x122 bytes) 000253[0] 1778535122.555574 write 0x0770eb thru 0x0779bd (0x8d3 bytes) 000254[0] 1778535122.611820 mapread 0x13adef thru 0x13bf86 (0x1198 bytes) 000255[1] 1778535122.655440 read 0x1309eb thru 0x1321b7 (0x17cd bytes) 000257[0] 1778535122.705280 write 0x0debf7 thru 0x0df85b (0xc65 bytes) 000260[1] 1778535122.778113 trunc from 0x9d8843 to 0x83d4bf (0x19b383 bytes) 000268[1] 1778535122.876433 mapwrite 0x08da48 thru 0x08efec (0x15a5 bytes) 000272[1] 1778535123.046657 mapread 0x348b19 thru 0x349812 (0xcfa bytes) 000273[0] 1778535123.047717 read 0x2d41fb thru 0x2d5d7d (0x1b83 bytes) 000276[1] 1778535123.147202 mapread 0x31f4fa thru 0x31f78c (0x293 bytes) 000277[1] 1778535123.174709 read 0x2da4f9 thru 0x2dc2eb (0x1df3 bytes) 000279[0] 1778535123.272115 mapwrite 0x286f54 thru 0x2874a2 (0x54f bytes) 000284[1] 1778535123.512908 trunc from 0x83d4c0 to 0x4e278f (0x35ad30 bytes) 000285[0] 1778535123.541363 trunc from 0x4e2790 to 0x1de2db (0x3044b4 bytes) 000286[0] 1778535123.601591 trunc from 0x1de2dc to 0x060bfc (0x17d6df bytes) 000287[0] 1778535123.642330 mapread 0x05bfa2 thru 0x05c2c2 (0x321 bytes) 000290[0] 1778535123.685360 trunc from 0x060bfd to 0x0884d3 (0x278d7 bytes) 000291[0] 1778535123.703071 mapread 0x061cde thru 0x062075 (0x398 bytes) 000293[0] 1778535123.770570 mapread 0x0375c6 thru 0x039318 (0x1d53 bytes) 000300[0] 1778535123.873239 read 0x008894 thru 0x008ca8 (0x415 bytes) 000302[1] 1778535123.903714 mapread 0x00b42d thru 0x00bf6b (0xb3f bytes) 000304[1] 1778535123.959110 mapwrite 0x9c97d8 thru 0x9cb19b (0x19c4 bytes) 000305[0] 1778535124.072214 mapread 0x637d33 thru 0x6380f6 (0x3c4 bytes) 000306[0] 1778535124.126674 read 0x0e55aa thru 0x0e5834 (0x28b bytes) 000308[1] 1778535124.158932 trunc from 0x9cb19c to 0x826dda (0x1a43c1 bytes) 000309[1] 1778535124.207207 write 0x9c32e6 thru 0x9c443b (0x1156 bytes) 000310[1] 1778535124.240955 read 0x0a1a07 thru 0x0a285c (0xe56 bytes) 000312[1] 1778535124.393442 read 0x342984 thru 0x343319 (0x996 bytes) 000313[1] 1778535124.409412 mapread 0x67d118 thru 0x67d3d7 (0x2c0 bytes) 000315[0] 1778535124.413628 read 0x936a3e thru 0x93869f (0x1c62 bytes) 000318[1] 1778535124.491265 mapwrite 0x60a9aa thru 0x60b000 (0x657 bytes) 000319[0] 1778535124.543108 read 0x48b20a thru 0x48c40a (0x1201 bytes) 000320[0] 1778535124.604724 write 0x420526 thru 0x42248f (0x1f6a bytes) 000321[0] 1778535124.645144 mapwrite 0x6119de thru 0x611f98 (0x5bb bytes) 000322[0] 1778535124.689852 read 0x3ef24f thru 0x3ef355 (0x107 bytes) 000323[0] 1778535124.742752 read 0x100996 thru 0x10245a (0x1ac5 bytes) 000327[0] 1778535124.792084 trunc from 0x9c443c to 0x558bbd (0x46b87e bytes) 000329[1] 1778535124.864658 trunc from 0x558bbe to 0x5716bc (0x18aff bytes) 000330[1] 1778535124.923904 trunc from 0x5716bd to 0x7d8455 (0x266d99 bytes) 000331[0] 1778535124.974713 mapwrite 0x1e553c thru 0x1e6c87 (0x174c bytes) 000332[0] 1778535125.068568 mapread 0x2e335e thru 0x2e3f6a (0xc0d bytes) 000333[0] 1778535125.104861 write 0x1695f2 thru 0x16a4f7 (0xf06 bytes) 000336[0] 1778535125.115049 mapread 0x051a33 thru 0x051cfe (0x2cc bytes) 000337[1] 1778535125.121733 trunc from 0x7d8456 to 0x637084 (0x1a13d1 bytes) 000339[0] 1778535125.192718 write 0x250d71 thru 0x252c47 (0x1ed7 bytes) 000340[1] 1778535125.228599 mapread 0x505216 thru 0x506463 (0x124e bytes) 000342[1] 1778535125.297556 write 0x3d8fbe thru 0x3d9eb1 (0xef4 bytes) 000343[0] 1778535125.334327 mapread 0x60dd90 thru 0x60f35e (0x15cf bytes) 000345[0] 1778535125.400464 mapread 0x50d48c thru 0x50f1c1 (0x1d36 bytes) 000347[1] 1778535125.455449 mapwrite 0x3fac24 thru 0x3fafe0 (0x3bd bytes) 000349[0] 1778535125.554283 read 0x491b3c thru 0x493306 (0x17cb bytes) 000350[1] 1778535125.582513 trunc from 0x637085 to 0x482d2c (0x1b4358 bytes) 000352[1] 1778535125.631932 mapread 0x327d24 thru 0x328f38 (0x1215 bytes) 000354[0] 1778535125.655927 trunc from 0x482d2d to 0x2ce7f2 (0x1b453a bytes) 000355[0] 1778535125.682382 mapread 0x222103 thru 0x223cfe (0x1bfc bytes) 000356[0] 1778535125.713691 read 0x0072a5 thru 0x007f46 (0xca2 bytes) 000357[1] 1778535125.783507 trunc from 0x2ce7f3 to 0x085c99 (0x248b59 bytes) 000360[0] 1778535125.873974 read 0x02aac4 thru 0x02b5cf (0xb0c bytes) 000361[1] 1778535125.905836 trunc from 0x085c9a to 0x063725 (0x22574 bytes) 000362[1] 1778535125.930329 mapread 0x01003d thru 0x0106eb (0x6af bytes) 000363[1] 1778535125.951839 write 0x7907a2 thru 0x791e6d (0x16cc bytes) 000365[1] 1778535125.955080 trunc from 0x791e6e to 0x60ca16 (0x185457 bytes) 000366[1] 1778535125.975827 mapwrite 0x247951 thru 0x247d0f (0x3bf bytes) 000368[0] 1778535126.061733 mapwrite 0x25e54b thru 0x25ed43 (0x7f9 bytes) 000370[1] 1778535126.183737 read 0x4ad790 thru 0x4ade07 (0x678 bytes) 000371[0] 1778535126.208665 trunc from 0x60ca17 to 0x283875 (0x3891a1 bytes) 000373[1] 1778535126.238734 mapread 0x08f673 thru 0x090ebf (0x184d bytes) 000375[0] 1778535126.270358 read 0x1b7890 thru 0x1b8390 (0xb01 bytes) 000377[1] 1778535126.325351 read 0x14ca4a thru 0x14d0d3 (0x68a bytes) 000378[1] 1778535126.367490 trunc from 0x283876 to 0x21679d (0x6d0d8 bytes) 000379[1] 1778535126.413599 trunc from 0x21679e to 0x61cbbd (0x406420 bytes) 000380[0] 1778535126.428128 read 0x133f35 thru 0x134788 (0x854 bytes) 000381[0] 1778535126.460480 write 0x0f5ced thru 0x0f65eb (0x8ff bytes) 000382[0] 1778535126.529319 write 0x963107 thru 0x96463c (0x1536 bytes) 000383[1] 1778535126.544485 write 0x3c2cfd thru 0x3c3c78 (0xf7c bytes) 000386[1] 1778535126.625831 write 0x7f6008 thru 0x7f7b62 (0x1b5b bytes) 000387[0] 1778535126.665351 mapwrite 0x92bdc3 thru 0x92d1ad (0x13eb bytes) 000388[0] 1778535126.863293 write 0x0f67f8 thru 0x0f7718 (0xf21 bytes) 000390[0] 1778535126.895266 mapwrite 0x844bc1 thru 0x844d37 (0x177 bytes) 000392[0] 1778535126.950668 read 0x4e57a4 thru 0x4e7409 (0x1c66 bytes) 000394[1] 1778535126.984476 mapwrite 0x5d0b91 thru 0x5d1fac (0x141c bytes) 000395[1] 1778535127.080691 write 0x01af6f thru 0x01afc4 (0x56 bytes) 000396[0] 1778535127.176403 mapread 0x82b7ed thru 0x82cb2f (0x1343 bytes) 000397[0] 1778535127.226213 trunc from 0x96463d to 0x57db08 (0x3e6b34 bytes) 000398[1] 1778535127.258072 write 0x715de5 thru 0x71778c (0x19a8 bytes) 000399[1] 1778535127.298395 write 0x1b2e5b thru 0x1b4d37 (0x1edd bytes) 000400[1] 1778535127.323571 mapread 0x5afcad thru 0x5b129b (0x15ef bytes) 000401[1] 1778535127.353368 read 0x2cecb0 thru 0x2cefe2 (0x333 bytes) 000402[0] 1778535127.411302 mapwrite 0x841392 thru 0x842d08 (0x1977 bytes) 000404[1] 1778535127.518955 trunc from 0x842d09 to 0x915f2f (0xd3227 bytes) 000405[0] 1778535127.561259 write 0x908404 thru 0x908c8f (0x88c bytes) 000406[1] 1778535127.592873 read 0x2f1cb2 thru 0x2f2acf (0xe1e bytes) 000407[1] 1778535127.615203 read 0x17a1ee thru 0x17bc8e (0x1aa1 bytes) 000408[0] 1778535127.722411 mapread 0x84bed4 thru 0x84ca88 (0xbb5 bytes) 000410[1] 1778535127.761879 write 0x2c27da thru 0x2c4721 (0x1f48 bytes) 000412[0] 1778535127.804520 mapread 0x17e0b9 thru 0x17f477 (0x13bf bytes) 000413[1] 1778535127.839604 read 0x189171 thru 0x189bda (0xa6a bytes) 000414[0] 1778535127.840196 read 0x127246 thru 0x1274a8 (0x263 bytes) 000416[1] 1778535127.901173 read 0x23c106 thru 0x23dceb (0x1be6 bytes) 000418[1] 1778535127.924316 trunc from 0x915f30 to 0x14b4c7 (0x7caa68 bytes) 000419[1] 1778535127.957653 trunc from 0x14b4c8 to 0x5158d1 (0x3ca40a bytes) 000421[0] 1778535127.972958 mapwrite 0x779e8b thru 0x779f71 (0xe7 bytes) 000423[0] 1778535128.173355 write 0x8ea4fc thru 0x8ebd16 (0x181b bytes) 000424[0] 1778535128.188609 read 0x6e8c23 thru 0x6e96ab (0xa89 bytes) 000425[0] 1778535128.210144 trunc from 0x8ebd17 to 0x2d105c (0x61acba bytes) 000430[0] 1778535128.261693 read 0x0be2f5 thru 0x0bfab7 (0x17c3 bytes) 000431[0] 1778535128.281747 write 0x348e0b thru 0x349400 (0x5f6 bytes) 000432[0] 1778535128.284463 mapread 0x075615 thru 0x0761bf (0xbab bytes) 000433[1] 1778535128.319380 write 0x97084e thru 0x9718df (0x1092 bytes) 000434[0] 1778535128.353366 mapwrite 0x886170 thru 0x887e04 (0x1c95 bytes) 000435[1] 1778535128.445649 write 0x46c555 thru 0x46db39 (0x15e5 bytes) 000436[1] 1778535128.458960 write 0x645fb2 thru 0x647cdc (0x1d2b bytes) 000439[1] 1778535128.472897 trunc from 0x9718e0 to 0x6bf8c5 (0x2b201a bytes) 000441[0] 1778535128.512091 mapread 0x42dc09 thru 0x42e233 (0x62b bytes) 000445[1] 1778535128.590440 read 0x5c64d1 thru 0x5c69f3 (0x523 bytes) 000448[0] 1778535128.632736 write 0x71e1e3 thru 0x71eed4 (0xcf2 bytes) 000449[1] 1778535128.652697 read 0x1dc731 thru 0x1de64d (0x1f1d bytes) 000452[1] 1778535128.725976 trunc from 0x71eed5 to 0x2ee447 (0x430a8d bytes) 000454[1] 1778535128.789735 read 0x1264cb thru 0x126953 (0x489 bytes) 000456[1] 1778535128.870323 mapread 0x1a0e61 thru 0x1a18c4 (0xa64 bytes) 000459[0] 1778535128.912235 write 0x59d103 thru 0x59eb6b (0x1a69 bytes) 000460[0] 1778535128.938452 write 0x7e32af thru 0x7e4aae (0x1800 bytes) 000461[1] 1778535128.952845 trunc from 0x7e4aaf to 0x737773 (0xad33b bytes) 000462[1] 1778535128.983870 trunc from 0x737774 to 0x40f52e (0x328245 bytes) 000463[0] 1778535128.993104 write 0x01b182 thru 0x01c132 (0xfb1 bytes) 000465[0] 1778535129.054562 trunc from 0x40f52f to 0x772655 (0x363127 bytes) 000466[1] 1778535129.078914 write 0x756488 thru 0x756ecd (0xa46 bytes) 000473[0] 1778535129.221280 read 0x03cdfa thru 0x03e6cf (0x18d6 bytes) 000474[0] 1778535129.269561 mapread 0x47e570 thru 0x47efa3 (0xa34 bytes) 000475[0] 1778535129.301653 write 0x02f96b thru 0x0302e4 (0x97a bytes) 000476[1] 1778535129.335612 write 0x687cf6 thru 0x6894f5 (0x1800 bytes) 000479[1] 1778535129.423525 write 0x8543cd thru 0x856159 (0x1d8d bytes) 000483[0] 1778535129.441625 mapread 0x512bb1 thru 0x513da4 (0x11f4 bytes) 000485[1] 1778535129.521555 read 0x37c15c thru 0x37d72e (0x15d3 bytes) 000486[0] 1778535129.579555 read 0x0486c6 thru 0x048f04 (0x83f bytes) 000487[1] 1778535129.676597 write 0x690b78 thru 0x691438 (0x8c1 bytes) 000488[1] 1778535129.697343 mapread 0x091be1 thru 0x092032 (0x452 bytes) 000491[0] 1778535129.737953 mapread 0x344bb4 thru 0x346aa1 (0x1eee bytes) 000493[1] 1778535129.775229 mapwrite 0x094c8c thru 0x09524b (0x5c0 bytes) 000494[0] 1778535129.862035 mapread 0x0d95f9 thru 0x0da4b6 (0xebe bytes) 000496[0] 1778535129.892929 write 0x026d35 thru 0x028377 (0x1643 bytes) 000498[1] 1778535129.921704 trunc from 0x85615a to 0x06ca93 (0x7e96c6 bytes) 000499[0] 1778535129.992209 read 0x025561 thru 0x0263e7 (0xe87 bytes) 000500[0] 1778535130.031028 write 0x8b7b1a thru 0x8b9050 (0x1537 bytes) All operations completed A-OK! PASS 16b (20s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 17:32:13 (1778535133) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 17:32:15 (1778535135) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.234138 s, 9.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.173123 s, 12.1 MB/s ldlm.namespaces.MGC192.168.204.120@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff915f42c40000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff915f457d0800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff915f457d0800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff915f457d0800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.11398 s, 17.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.79549 s, 58.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.29554 s, 80.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.16105 s, 33.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.53941 s, 23.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.90698 s, 55.0 MB/s PASS 16d (26s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 17:32:41 (1778535161) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.158069 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836082 s, 12.5 MB/s PASS 16e (4s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 17:32:45 (1778535165) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 17:33:09 (1778535189) TIMEOUT PASS 16g (24s) == sanityn test 16i: read after truncate file ============ 17:33:33 (1778535213) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (3s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 17:33:36 (1778535216) fail_loc=0x8000030a PASS 17 (6s) == sanityn test 18: mmap sanity check =========================================================================================== 17:33:42 (1778535222) mmap test1: basic mmap operation (PASS, 0.063945s) mmap test2: MAP_PRIVATE not write back (PASS, 0.04223s) mmap test3: concurrent mmap ops on two nodes (PASS, 11.383s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.2247s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.078408s) mmap test6: check mmap write/read content on two nodes (PASS, 0.12451s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0521s) PASS 18 (20s) == sanityn test 19: test concurrent uncached read races ========================================================================= 17:34:02 (1778535242) oleg420-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg420-client: oleg420-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 17:34:04 (1778535244) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 17:34:06 (1778535246) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (3s) == sanityn test 23: others should see updated atime while another read============================================================== 17:34:09 (1778535249) now is 1778535250 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 new atime is 1778535311 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 17:35:14 (1778535314) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (4s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 17:35:18 (1778535318) PASS 24b (4s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 17:35:22 (1778535322) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 17:35:25 (1778535325) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (2s) == sanityn test 26a: allow mtime to get older ============ 17:35:27 (1778535327) utime: good mknod times 1778535327 <= 1778535328 <= 1778535328 for /mnt/lustre/f26a utime: good mknod times 1778535327 <= 1778535328 <= 1778535328 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 17:35:32 (1778535332) PASS 26b (7s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 17:35:40 (1778535340) 0 PASS 30 (6s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 17:35:46 (1778535346) fail_loc=0x314 PASS 31a (5s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 17:35:51 (1778535351) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg420-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.120@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (15s) == sanityn test 31r: open-rename(replace) race =========== 17:36:06 (1778535366) -rw-r--r-- 1 root root 0 May 11 17:36 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (7s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 17:36:15 (1778535375) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 17:36:17 (1778535377) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 17:36:19 (1778535379) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 17:36:20 (1778535380) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (2s) == sanityn test 34: no lock timeout under IO ============= 17:36:22 (1778535382) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg420-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff915f42c40000.ost_server_uuid,osc.lustre-OST0000-osc-ffff915f457d0800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff915f42c40000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff915f457d0800.ost_server_uuid in IDLE state after 0 sec oleg420-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff915f42c40000.ost_server_uuid,osc.lustre-OST0001-osc-ffff915f457d0800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff915f42c40000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff915f457d0800.ost_server_uuid in FULL state after 0 sec oleg420-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff915f42c40000.ost_server_uuid,osc.lustre-OST0000-osc-ffff915f457d0800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff915f42c40000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff915f457d0800.ost_server_uuid in IDLE state after 0 sec oleg420-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff915f42c40000.ost_server_uuid,osc.lustre-OST0001-osc-ffff915f457d0800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff915f42c40000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff915f457d0800.ost_server_uuid in FULL state after 0 sec oleg420-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff915f42c40000.ost_server_uuid,osc.lustre-OST0000-osc-ffff915f457d0800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff915f42c40000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff915f457d0800.ost_server_uuid in IDLE state after 0 sec oleg420-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff915f42c40000.ost_server_uuid,osc.lustre-OST0001-osc-ffff915f457d0800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff915f42c40000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff915f457d0800.ost_server_uuid in FULL state after 0 sec PASS 34 (91s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 17:37:53 (1778535473) fail_loc=0x80000317 Race attempt 0 Wait for 44641 44742 for 60 sec... - open/close 1356 (time 1778535486.73 total 10.01 last 135.46) - open/close 2633 (time 1778535496.73 total 20.01 last 127.69) total: 4000 open/close in 29.54 seconds: 135.39 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.120@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre 192.168.204.120@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 17:38:59 (1778535539) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg420-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.623673 s, 84.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.542412 s, 96.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.592372 s, 88.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.564986 s, 92.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7525376) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.521959 s, 100 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7523328) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.487148 s, 108 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.523953 s, 100 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.444078 s, 118 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.48873 s, 107 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.614151 s, 85.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.516591 s, 101 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7525376) after_dd(7474176) after(7525376) PASS 36 (256s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 17:43:15 (1778535795) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 - create 5073 (time 1778535807.13 total 10.00 last 507.25) - create 9264 (time 1778535817.13 total 20.00 last 418.98) total: 10000 create in 21.87 seconds: 457.19 ops/second PASS 37 (27s) == sanityn test 39a: test from 11063 ============================================================================================ 17:43:42 (1778535822) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 17:43:45 (1778535825) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 17:43:48 (1778535828) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 17:43:52 (1778535832) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 40a: pdirops: create vs others ======================================================================== 17:43:55 (1778535835) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 17:44:04 (1778535844) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (9s) == sanityn test 40c: pdirops: link and others ======================================================================== 17:44:13 (1778535853) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (9s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 17:44:22 (1778535862) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (9s) == sanityn test 40e: pdirops: rename and others ======================================================================== 17:44:31 (1778535871) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (9s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 17:44:40 (1778535880) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (6s) == sanityn test 41b: pdirops: create vs create ======================================================================== 17:44:46 (1778535886) open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 17:44:52 (1778535892) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 17:44:58 (1778535898) Conflict PASS 41d (6s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 17:45:04 (1778535904) Conflict PASS 41e (6s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 17:45:10 (1778535910) Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 17:45:16 (1778535916) Conflict PASS 41g (6s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 17:45:22 (1778535922) Conflict PASS 41h (7s) == sanityn test 41i: reint_open: create vs create ======== 17:45:29 (1778535929) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (853s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 17:59:42 (1778536782) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (13s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 17:59:55 (1778536795) open: File exists Conflict PASS 42b (12s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 18:00:07 (1778536807) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (12s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 18:00:19 (1778536819) Conflict PASS 42d (13s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 18:00:32 (1778536832) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (14s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 18:00:46 (1778536846) Conflict PASS 42f (14s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 18:01:00 (1778536860) Conflict PASS 42g (12s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 18:01:12 (1778536872) Conflict PASS 42h (11s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 18:01:23 (1778536883) PASS 43a (53s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 18:02:16 (1778536936) Conflict PASS 43b (12s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 18:02:28 (1778536948) Conflict PASS 43c (13s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 18:02:41 (1778536961) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (12s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 18:02:53 (1778536973) Conflict PASS 43e (12s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 18:03:05 (1778536985) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (10s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 18:03:15 (1778536995) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (11s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 18:03:26 (1778537006) Conflict PASS 43h (11s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 18:03:38 (1778537018) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (3s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 18:03:40 (1778537020) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (97s) == sanityn test 43k: unlink vs create ==================== 18:05:17 (1778537117) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (786s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 18:18:23 (1778537903) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon May 11 18:18:28 EDT 2026 PASS 44a (7s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 18:18:30 (1778537910) open: File exists Conflict PASS 44b (8s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 18:18:38 (1778537918) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (8s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 18:18:46 (1778537926) Conflict PASS 44d (8s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 18:18:54 (1778537934) Conflict PASS 44e (7s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 18:19:01 (1778537941) Conflict PASS 44f (7s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 18:19:08 (1778537948) Conflict PASS 44g (7s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 18:19:15 (1778537955) Conflict PASS 44h (7s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 18:19:22 (1778537962) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 18:19:23 (1778537963) PASS 45a (59s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 18:20:22 (1778538022) Conflict PASS 45b (7s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 18:20:29 (1778538029) Conflict PASS 45c (7s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 18:20:36 (1778538036) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (8s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 18:20:44 (1778538044) Conflict PASS 45e (8s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 18:20:52 (1778538052) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (8s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 18:21:00 (1778538060) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (8s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 18:21:08 (1778538068) Conflict PASS 45h (8s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 18:21:16 (1778538076) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 18:21:17 (1778538077) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (578s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 18:30:55 (1778538655) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (7s) == sanityn test 46b: pdirops: link vs create ======================================================================== 18:31:02 (1778538662) open: File exists Conflict PASS 46b (6s) == sanityn test 46c: pdirops: link vs link ======================================================================== 18:31:08 (1778538668) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 18:31:14 (1778538674) Conflict PASS 46d (7s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 18:31:21 (1778538681) Conflict PASS 46e (6s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 18:31:27 (1778538687) Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 18:31:33 (1778538693) Conflict PASS 46g (7s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 18:31:40 (1778538700) Conflict PASS 46h (6s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 18:31:47 (1778538707) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 18:31:48 (1778538708) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 18:31:49 (1778538709) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 18:31:50 (1778538710) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 18:31:52 (1778538712) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 18:31:53 (1778538713) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 18:31:55 (1778538715) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 18:31:56 (1778538716) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 18:31:57 (1778538717) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00934458 s, 1.1 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 18:32:05 (1778538725) 0+1 records in 0+1 records out 158 bytes copied, 0.0107447 s, 14.7 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242678) - No such process PASS 51a (5s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 18:32:10 (1778538730) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0102549 s, 99.9 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231385 s, 45.3 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 18:32:25 (1778538745) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00701656 s, 146 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 18:32:32 (1778538752) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156427 s, 67.0 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 18:32:36 (1778538756) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2430 0x97e 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2430 0x97e 0 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 18:32:41 (1778538761) llite.lustre-ffff915f42c40000.xattr_cache=1 llite.lustre-ffff915f457d0800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:42.000000000 -0400 Modify: 2026-05-11 18:32:42.000000000 -0400 Change: 2026-05-11 18:32:42.000000000 -0400 Birth: 2026-05-11 18:32:42.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:48.000000000 -0400 Modify: 2026-05-11 18:32:48.000000000 -0400 Change: 2026-05-11 18:32:48.000000000 -0400 Birth: 2026-05-11 18:32:47.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:53.000000000 -0400 Modify: 2026-05-11 18:32:53.000000000 -0400 Change: 2026-05-11 18:32:53.000000000 -0400 Birth: 2026-05-11 18:32:53.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:53.000000000 -0400 Modify: 2026-05-11 18:32:54.000000000 -0400 Change: 2026-05-11 18:32:54.000000000 -0400 Birth: 2026-05-11 18:32:53.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:59.000000000 -0400 Modify: 2026-05-11 18:32:59.000000000 -0400 Change: 2026-05-11 18:32:59.000000000 -0400 Birth: 2026-05-11 18:32:59.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:59.000000000 -0400 Modify: 2026-05-11 18:32:59.000000000 -0400 Change: 2026-05-11 18:32:59.000000000 -0400 Birth: 2026-05-11 18:32:59.000000000 -0400 PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 18:33:07 (1778538787) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 18:33:15 (1778538795) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (9s) == sanityn test 55c: rename vs unlink orphan target dir == 18:33:23 (1778538803) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 18:33:37 (1778538817) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 60: Verify data_version behaviour ======== 18:33:47 (1778538827) PASS 60 (3s) == sanityn test 70a: cd directory && rm directory ======== 18:33:50 (1778538830) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 18:33:53 (1778538833) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 18:33:55 (1778538835) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00546643 s, 12.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 18:33:57 (1778538837) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00525411 s, 12.5 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 18:33:58 (1778538838) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 18:34:00 (1778538840) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00668038 s, 9.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 18:34:01 (1778538841) llite.lustre-ffff915f42c40000.xattr_cache=1 llite.lustre-ffff915f457d0800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 18:34:04 (1778538844) llite.lustre-ffff915f42c40000.xattr_cache=1 llite.lustre-ffff915f457d0800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 18:34:07 (1778538847) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0045471 s, 225 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0152153 s, 67.3 kB/s 252714: get lock2 252715: get lock1 252715: done 252715: get lock2 252714: done 252714: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 252714: cannot get lock1: Resource deadlock avoided 252715: done 252715: exit rc=0 252714: done 252714: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 18:34:14 (1778538854) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0377347 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167353 s, 62.7 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253289 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (9s) == sanityn test 76: Verify MDT open_files listing ======== 18:34:22 (1778538862) opening files: ........ closing files: ........ PASS 76 (24s) == sanityn test 77a: check FIFO NRS policy =============== 18:34:46 (1778538886) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.174112 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309805 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442409 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559333 s, 18.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804915 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.064844 s, 16.2 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108505 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098663 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933493 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737096 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982971 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088516 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102105 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932993 s, 11.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102566 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099687 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111833 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214873 s, 48.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239664 s, 43.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175021 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017802 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184429 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244964 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266562 s, 39.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200597 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450547 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258652 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224735 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228756 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279393 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029315 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266382 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322706 s, 32.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=4 removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (4s) == sanityn test 77b: check CRR-N NRS policy ============== 18:34:50 (1778538890) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.162041 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231703 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410322 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396523 s, 26.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107749 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990091 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114898 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929102 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095736 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987137 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124633 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102856 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135798 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129594 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141108 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126374 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151125 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01886 s, 55.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416872 s, 252 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254925 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270633 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0031793 s, 330 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286775 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288369 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0430305 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380324 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359489 s, 29.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590049 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032878 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438312 s, 23.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450752 s, 23.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222961 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0452876 s, 23.2 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.140624 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425705 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308504 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554589 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544886 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681341 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708858 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499503 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775975 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823807 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961622 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892007 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867569 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111685 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917533 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837005 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107256 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111176 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241195 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006003 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226508 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020617 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100015 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263276 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02359 s, 44.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183447 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224318 s, 46.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144108 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276058 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025542 s, 41.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292742 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688827 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602878 s, 174 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 18:34:55 (1778538895) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.172697 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152504 s, 68.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603941 s, 17.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645444 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649273 s, 16.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632045 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687612 s, 15.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969301 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809859 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849346 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936649 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826418 s, 12.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862891 s, 12.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908675 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747599 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779536 s, 13.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903745 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101728 s, 103 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163077 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119181 s, 88.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177138 s, 59.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152317 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181499 s, 57.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020419 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197032 s, 53.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189207 s, 55.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202658 s, 51.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386503 s, 27.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218536 s, 48.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333608 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234203 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248802 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029399 s, 35.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.16109 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517904 s, 20.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563167 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385723 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663133 s, 15.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523642 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659761 s, 15.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585267 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559299 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549211 s, 19.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770347 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11916 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114244 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12594 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116903 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130667 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142987 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166149 s, 63.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256931 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237018 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316341 s, 33.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227796 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324383 s, 32.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315674 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026368 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229421 s, 45.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297448 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357738 s, 29.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217329 s, 48.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414395 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479746 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034897 s, 30.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0390012 s, 26.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.141488 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410439 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053068 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590696 s, 17.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857542 s, 12.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700572 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925446 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665958 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992317 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931349 s, 11.3 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868914 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736011 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115385 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926966 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847472 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638833 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061483 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229344 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173759 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181663 s, 57.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135753 s, 77.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309058 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189831 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223856 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330347 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031272 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126217 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270153 s, 38.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368765 s, 28.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018342 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312706 s, 33.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313261 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337106 s, 31.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (7s) == sanityn test 77d: check TRR nrs policy ================ 18:35:02 (1778538902) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.218438 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511712 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535678 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0460566 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442745 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557491 s, 18.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725343 s, 14.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867105 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610037 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845362 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807018 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106216 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944204 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100011 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114453 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118145 s, 8.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109117 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228287 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279087 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126181 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230986 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224061 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238919 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252174 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245145 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198373 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391507 s, 26.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235775 s, 44.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305452 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00407999 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273877 s, 38.3 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332198 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0351159 s, 29.9 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.188147 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644334 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648465 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101955 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105181 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119434 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119317 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108114 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117456 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129161 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119858 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123822 s, 8.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125579 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113797 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136193 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12447 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121246 s, 8.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309006 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229941 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224317 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020961 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249928 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182328 s, 57.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263537 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263287 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258756 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301919 s, 34.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025289 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265653 s, 39.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037049 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00442212 s, 237 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259778 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00307129 s, 341 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.18594 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346632 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420904 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090035 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0839064 s, 12.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110778 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118941 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121858 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126433 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112678 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114712 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129742 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105226 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128275 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109709 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129019 s, 8.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124601 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101104 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290569 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219819 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220631 s, 47.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114224 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235472 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253051 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195335 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197654 s, 53.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312424 s, 33.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267755 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271474 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379948 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288379 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357998 s, 29.3 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (6s) == sanityn test 77e: check TBF NID nrs policy ============ 18:35:08 (1778538908) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.20@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.206657 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270088 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318744 s, 32.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0757489 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072571 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922456 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774196 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830159 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11265 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107614 s, 9.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110227 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11547 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109421 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113232 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109528 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109277 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127309 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591183 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119973 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217263 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812233 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609652 s, 17.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557412 s, 18.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630676 s, 16.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667845 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770044 s, 13.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509883 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077717 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00307101 s, 341 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845895 s, 12.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816198 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801456 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799361 s, 13.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.172674 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226912 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444844 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484798 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257701 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815933 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821264 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075356 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828659 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713483 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074461 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693795 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718139 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689706 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0915776 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988814 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108112 s, 9.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317257 s, 33.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468798 s, 22.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042467 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446013 s, 23.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446332 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621231 s, 16.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472194 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499978 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597952 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504587 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656907 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523367 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702832 s, 14.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507314 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557395 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055817 s, 18.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.162585 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030917 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056392 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682393 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875988 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06375 s, 16.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744739 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790599 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931203 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885408 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084954 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835304 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101497 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117642 s, 8.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122637 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110826 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132712 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027969 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253283 s, 41.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243562 s, 43.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212727 s, 49.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195261 s, 53.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343071 s, 30.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202533 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234732 s, 44.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264386 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441887 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230757 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284426 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449052 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391689 s, 26.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302912 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047847 s, 21.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.150687 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379067 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493607 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470803 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629413 s, 16.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726854 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616127 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603473 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102869 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919426 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954532 s, 11.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10945 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119172 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107015 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113583 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116947 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104948 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188304 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194144 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134462 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288863 s, 36.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272398 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223681 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302163 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188595 s, 55.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214527 s, 48.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187927 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262717 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289189 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025733 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268458 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275212 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029714 s, 35.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=3 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (11s) == sanityn test 77f: check TBF JobID nrs policy ========== 18:35:19 (1778538919) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.224717 s, 74.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347791 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229559 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395864 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454813 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425604 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436548 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980188 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932664 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106873 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117849 s, 8.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113912 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114136 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988188 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127749 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878545 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100648 s, 10.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164577 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163528 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262751 s, 39.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298284 s, 35.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184415 s, 56.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221081 s, 47.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236131 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186332 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202822 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214843 s, 48.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196142 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254632 s, 41.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253193 s, 41.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273136 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251253 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268232 s, 39.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.164687 s, 102 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417806 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462242 s, 22.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0467598 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202089 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449146 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596803 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453571 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471045 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352455 s, 29.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597262 s, 17.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976285 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765826 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108723 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118957 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106301 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113236 s, 9.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142939 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115116 s, 91.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233818 s, 44.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173737 s, 60.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314328 s, 33.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374106 s, 28.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181534 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182528 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189788 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224483 s, 46.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245426 s, 42.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251417 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287219 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285544 s, 36.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293021 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316807 s, 33.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.175025 s, 95.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736868 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755066 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427561 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402214 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854333 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982197 s, 10.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422458 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106254 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119358 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121798 s, 8.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110155 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0955438 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118339 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177015 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182199 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142042 s, 7.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00503022 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190445 s, 55.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055467 s, 189 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979734 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330247 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227534 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02695 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108041 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298881 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360816 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237387 s, 44.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372615 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757664 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392328 s, 26.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558281 s, 18.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276461 s, 37.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=1 removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.184723 s, 90.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327858 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403606 s, 26.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115586 s, 9.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114647 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115362 s, 9.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121171 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0923292 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107895 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111004 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892433 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109342 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103307 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104307 s, 10.1 MB/sldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123704 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132682 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125894 s, 8.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210443 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114038 s, 91.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198778 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265074 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219044 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149958 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166097 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175863 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221567 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429916 s, 24.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223556 s, 46.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198345 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246591 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228585 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236568 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268037 s, 39.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (9s) == sanityn test 77g: Change TBF type directly ============ 18:35:28 (1778538928) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (5s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 18:35:33 (1778538933) oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg420-client: oleg420-server: ssh exited with exit code 2 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg420-client: oleg420-server: ssh exited with exit code 12 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg420-client: oleg420-server: ssh exited with exit code 12 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg420-client: oleg420-server: ssh exited with exit code 12 PASS 77h (6s) == sanityn test 77i: Change rank of TBF rule ============= 18:35:39 (1778538939) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 18:35:48 (1778538948) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.223803 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711601 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748785 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831457 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657362 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101404 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0896814 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830102 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110219 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104274 s, 10.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102309 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106929 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141678 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207466 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222124 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313284 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323915 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200221 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218134 s, 48.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177128 s, 59.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269755 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217905 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205404 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156776 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228892 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263978 s, 39.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247927 s, 42.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256043 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234437 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288777 s, 36.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023824 s, 44.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261276 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750213 s, 140 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95579 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4862 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (53s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 18:36:41 (1778539001) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.673743 s, 24.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145422 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157539 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179602 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178522 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1943 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174207 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202898 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195956 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3435 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.350039 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372474 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386846 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359152 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385892 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37377 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376033 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131791 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132213 s, 7.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138024 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134093 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140964 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136869 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138259 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136731 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137078 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136112 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13908 s, 7.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137808 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138504 s, 7.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136992 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1392 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14495 s, 7.2 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7811 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9531 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.670328 s, 25.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13808 s, 7.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148365 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15296 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147852 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171438 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165736 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162053 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189865 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352976 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393811 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378365 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.412043 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395812 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.418906 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.403384 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.431873 s, 2.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11393 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111246 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122354 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11301 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110388 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110378 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113061 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112763 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113279 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11369 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115943 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116314 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116148 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118003 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119605 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121491 s, 8.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7391 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9297 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (135s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 18:38:56 (1778539136) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.221583 s, 75.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232627 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034997 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632273 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870903 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0914093 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101561 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927281 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093379 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112029 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101274 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117144 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10358 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969056 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981348 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927663 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110357 s, 9.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234479 s, 44.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159123 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338074 s, 31.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107692 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0043045 s, 244 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00501948 s, 209 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00474915 s, 221 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583173 s, 180 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239221 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303092 s, 34.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207809 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00357506 s, 293 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178535 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612399 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607451 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00303352 s, 346 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.99829 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77232 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.20@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.20@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.169076 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298106 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.054435 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497471 s, 21.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851369 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724787 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090534 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753823 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0996524 s, 10.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103488 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099989 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115645 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105037 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132655 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143459 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217547 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242355 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168039 s, 62.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214851 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192728 s, 54.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278884 s, 37.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254314 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242002 s, 43.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254936 s, 41.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253036 s, 41.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199135 s, 52.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212956 s, 49.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275668 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205758 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214206 s, 49.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262248 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258069 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297551 s, 35.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.01083 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76009 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.20@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.177746 s, 94.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359329 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456931 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0491438 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146477 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147619 s, 7.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153889 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156288 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16009 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145487 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147045 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143886 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157685 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16475 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151302 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140315 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165981 s, 6.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224246 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181147 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292968 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315127 s, 33.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242439 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237617 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240305 s, 43.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265491 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269229 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306878 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031333 s, 33.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328552 s, 31.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316915 s, 33.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338545 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341108 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032856 s, 31.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97119 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92036 s, 21.3 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.282689 s, 59.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271185 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387315 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490115 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315391 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299234 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563784 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365939 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305485 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134751 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142955 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175835 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150579 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331435 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319367 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333941 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335519 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142994 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247632 s, 42.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00494595 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00469282 s, 223 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908421 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00467539 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263083 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398815 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018809 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783526 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343939 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00356293 s, 294 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039903 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476764 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412207 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928996 s, 113 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.85347 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76905 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96531 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77525 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.473623 s, 35.4 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0384016 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744382 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682159 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663311 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0814851 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095448 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102762 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093039 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333158 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318971 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332922 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341859 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352751 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341999 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372308 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380165 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028415 s, 36.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338732 s, 31.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300223 s, 34.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316296 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343687 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389161 s, 26.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343511 s, 30.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363758 s, 28.8 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359121 s, 29.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383012 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034446 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365723 s, 28.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391487 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366436 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403266 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401318 s, 26.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5226 s, 5.4 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4554 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.181414 s, 92.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299423 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338616 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894283 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877355 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103772 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101911 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111261 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110238 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106224 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111638 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11976 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998051 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907446 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961946 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105128 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11122 s, 9.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266618 s, 39.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250841 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02373 s, 44.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205441 s, 51.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024052 s, 43.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236042 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260702 s, 40.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250903 s, 41.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231642 s, 45.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278898 s, 37.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296706 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247207 s, 42.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256107 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030063 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255236 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295116 s, 35.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.05869 s, 20.7 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76676 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.391328 s, 42.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440914 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341347 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108957 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103881 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143694 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131514 s, 8.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132646 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146588 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159883 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159462 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177545 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195117 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343671 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331038 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.43472 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.465459 s, 2.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027347 s, 38.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334631 s, 31.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039109 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283183 s, 37.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350384 s, 29.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455923 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427869 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314513 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402989 s, 26.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358984 s, 29.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395945 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401525 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427486 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431423 s, 24.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340691 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436932 s, 24.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.85243 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7696 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.84269 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77916 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (374s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 18:45:10 (1778539510) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (5s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 18:45:15 (1778539515) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg420-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.78819 s, 277 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg420-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0416 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (52s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 18:46:07 (1778539567) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.162006 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388945 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356186 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537516 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105189 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115778 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133649 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116565 s, 9.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957645 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125652 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129274 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12561 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125257 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138069 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135635 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125256 s, 8.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff915f42c40000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130222 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198338 s, 52.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015377 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220844 s, 47.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267576 s, 39.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154822 s, 67.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215552 s, 48.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315715 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246536 s, 42.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213775 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196186 s, 53.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284452 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227566 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648617 s, 162 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282211 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285545 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084655 s, 124 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff915f42c40000.lock_unused_count=1 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg420-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg420-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.09231 s, 20.6 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96508 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.223561 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279842 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442351 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905478 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992924 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116926 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106888 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119739 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138062 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140902 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149449 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14325 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139403 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13606 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125184 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214488 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24336 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234385 s, 44.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333218 s, 31.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294043 s, 35.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301046 s, 34.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337526 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261765 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305897 s, 34.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358253 s, 29.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335514 s, 31.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358319 s, 29.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326715 s, 32.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034173 s, 30.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343353 s, 30.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337075 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352838 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401887 s, 26.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg420-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg420-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.11457 s, 20.5 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.98079 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (70s) == sanityn test 77o: Changing rank should not panic ====== 18:47:17 (1778539637) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 18:47:21 (1778539641) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (43s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 18:48:04 (1778539684) mds.MDS.mdt.nrs_policies=tbf oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg420-client: oleg420-server: ssh exited with exit code 36 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg420-client: oleg420-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 18:48:17 (1778539697) oleg420-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg420-client: oleg420-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 18:48:20 (1778539700) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:48:21.000000000 -0400 Modify: 2026-05-11 18:48:21.000000000 -0400 Change: 2026-05-11 18:48:21.000000000 -0400 Birth: 2026-05-11 18:48:21.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 18:48:33 (1778539713) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 18:48:34 (1778539714) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 18:48:36 (1778539716) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 18:48:37 (1778539717) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 18:48:38 (1778539718) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 18:48:39 (1778539719) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7129 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7129 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 18:48:41 (1778539721) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 18:48:42 (1778539722) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 90: open/create and unlink striped directory ========================================================== 18:48:50 (1778539730) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 18:48:51 (1778539731) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 18:48:52 (1778539732) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 18:48:53 (1778539733) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2644 0xa54 0 1 2626 0xa42 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2645 0xa55 0 1 2627 0xa43 0 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 18:49:02 (1778539742) osc.lustre-OST0000-osc-ffff915f42c40000.idle_timeout=0 osc.lustre-OST0000-osc-ffff915f457d0800.idle_timeout=0 osc.lustre-OST0001-osc-ffff915f42c40000.idle_timeout=0 osc.lustre-OST0001-osc-ffff915f457d0800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.056801 s, 144 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09625 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 284126 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0183134 s, 224 kB/s osc.lustre-OST0000-osc-ffff915f42c40000.idle_timeout=debug osc.lustre-OST0000-osc-ffff915f457d0800.idle_timeout=debug osc.lustre-OST0001-osc-ffff915f42c40000.idle_timeout=debug osc.lustre-OST0001-osc-ffff915f457d0800.idle_timeout=debug PASS 94 (11s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 18:49:13 (1778539753) llite.lustre-ffff915f42c40000.fast_read=0 llite.lustre-ffff915f457d0800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00196981 s, 6.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff915f42c40000.fast_read=1 llite.lustre-ffff915f457d0800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 18:49:21 (1778539761) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 18:49:22 (1778539762) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00268062 s, 1.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 18:49:25 (1778539765) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.030297 s, 69.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 18:49:28 (1778539768) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0309406 s, 67.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 18:49:30 (1778539770) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00252272 s, 4.8 MB/s PASS 100e (3s) == sanityn test 101a: Discard DoM data on unlink ========= 18:49:33 (1778539773) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000621045 s, 6.6 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 18:49:35 (1778539775) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000780779 s, 5.2 MB/s PASS 101b (3s) == sanityn test 101c: Discard DoM data on close-unlink === 18:49:38 (1778539778) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000692417 s, 5.9 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 18:49:42 (1778539782) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1778539783 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1778539785 Finishing test test23 at 1778539786 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 18:49:49 (1778539789) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 18:49:51 (1778539791) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 18:50:39 (1778539839) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 18:50:40 (1778539840) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00307194 s, 333 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:50:41.000000000 -0400 Modify: 2026-05-11 18:50:41.000000000 -0400 Change: 2026-05-11 18:50:41.000000000 -0400 Birth: 2026-05-11 18:50:40.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1778539840 1778539841 1778539841 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:50:41.000000000 -0400 Modify: 2026-05-11 18:50:41.000000000 -0400 Change: 2026-05-11 18:50:41.000000000 -0400 Birth: 2026-05-11 18:50:40.000000000 -0400 1778539841 1024 1 PASS 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 18:50:42 (1778539842) PASS 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 18:50:45 (1778539845) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0966567 s, 108 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2653 0xa5d 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 18:50:49 (1778539849) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0959709 s, 109 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2632 0xa48 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 18:50:58 (1778539858) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00402806 s, 2.5 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.32548 s, 1.9 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 18:51:05 (1778539865) 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: detected 4 online CPUs by sysfs oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: Force libcfs to create 2 CPU partitions oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 34 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 192.168.204.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.120@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg420-client.virtnet /mnt/lustre (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg420-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg420-server: detected 4 online CPUs by sysfs oleg420-server: Force libcfs to create 2 CPU partitions Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre Starting client: oleg420-client.virtnet: -o user_xattr,flock oleg420-server@tcp:/lustre /mnt/lustre2 PASS 109 (598s) == sanityn test 110: do not grant another lock on resend ========================================================== 19:01:04 (1778540464) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 19:01:04.000000000 -0400 Modify: 2026-05-11 19:01:04.000000000 -0400 Change: 2026-05-11 19:01:04.000000000 -0400 Birth: 2026-05-11 19:01:04.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg420-client.virtnet oleg420-client.virtnet PASS 110 (38s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 19:01:42 (1778540502) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 19:01:43 (1778540503) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (2s) == sanityn test 113: check servers of specified fs ======= 19:01:45 (1778540505) lustre-OST0000-osc-ffff915f5ae16000 active. lustre-OST0001-osc-ffff915f5ae16000 active. lustre-OST0000-osc-ffff915f5ae16000 active. lustre-OST0001-osc-ffff915f5ae16000 active. MGC192.168.204.120@tcp active. PASS 113 (3s) cleanup: ====================================================== == sanityn test complete, duration 5720 sec ============== 19:01:49 (1778540509) Stopping clients: oleg420-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg420-client.virtnet /mnt/lustre2 opts: