-----============= acceptance-small: sanityn ============----- Thu Mar 26 08:38:11 EDT 2026 oleg260-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Started clients oleg260-client.virtnet: 192.168.202.160@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg260-client.virtnet: executing check_config_client /mnt/lustre oleg260-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg260-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff97ae03c84000.idle_timeout=debug osc.lustre-OST0000-osc-ffff97ae0a7fe800.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ae03c84000.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ae0a7fe800.idle_timeout=debug disable quota as required oleg260-server: oleg260-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254789 s, 41.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7141] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 08:38:49 (1774528729) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (5s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 08:38:54 (1774528734) -rw-r--r-- 1 root root 0 Mar 26 08:38 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (6s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 08:39:01 (1774528741) -rw-r--r-- 1 root root 0 Mar 26 08:39 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (5s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 08:39:07 (1774528747) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (6s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 08:39:12 (1774528752) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (5s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 08:39:17 (1774528757) total 1 -rwxrwxrwx 1 root root 0 Mar 26 08:38 f2a -rwxrwxrwx 1 root root 0 Mar 26 08:39 f2b total 1 -rwxrwxrwx 1 root root 0 Mar 26 08:38 f2a -rwxrwxrwx 1 root root 0 Mar 26 08:39 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.012854 s, 39.8 kB/s PASS 2e (6s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 08:39:23 (1774528763) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (3s) == sanityn test 2g: check blocks update on sync write ==== 08:39:26 (1774528766) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.274543 s, 7.6 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (6s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 08:39:32 (1774528772) PASS 3 (7s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 08:39:39 (1774528779) PASS 4 (7s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 08:39:46 (1774528786) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (6s) == sanityn test 6: remove of open file on other node ============================================================================ 08:39:52 (1774528792) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (6s) == sanityn test 7: remove of open directory on other node ======================================================================= 08:39:58 (1774528798) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (7s) == sanityn test 8: remove of open special file on other node ==================================================================== 08:40:05 (1774528805) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (6s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 08:40:11 (1774528811) PASS 9a (6s) == sanityn test 9b: append to striped sparse file ======== 08:40:18 (1774528818) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612082 s, 17.1 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0757079 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (7s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 08:40:24 (1774528824) 1+0 records in 1+0 records out 1 byte copied, 0.00454379 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0103884 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0103536 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00899137 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00698993 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00557572 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00519203 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00565866 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00787903 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00563047 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00270995 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00392234 s, 0.3 kB/s PASS 10a (6s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 08:40:31 (1774528831) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0121506 s, 253 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0553408 s, 74.0 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000273398 s, 11.2 MB/s PASS 10b (6s) == sanityn test 11: execution of file opened for write should return error ============================================================== 08:40:36 (1774528836) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (6s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 08:40:42 (1774528842) warning: '-runtime' deprecated, use '-t runtime' instead - link 428 (time 1774528853.94 total 10.00 last 42.80) - link 838 (time 1774528863.94 total 20.00 last 40.99) - link 1247 (time 1774528873.95 total 30.01 last 40.87) - link 1640 (time 1774528883.95 total 40.02 last 39.28) - link 2059 (time 1774528893.97 total 50.03 last 41.82) - link 2419 (time 1774528903.98 total 60.04 last 35.98) - link 2794 (time 1774528914.02 total 70.08 last 37.34) - link 3196 (time 1774528924.04 total 80.10 last 40.12) - link 3610 (time 1774528934.08 total 90.14 last 41.25) total: 4026 link in 100.01 seconds: 40.26 ops/second using seed 1701063374 running for 100 seconds - stat 10000 (time 1774528854 ; total 10 ; last 10) - stat 20000 (time 1774528863 ; total 19 ; last 9) - stat 30000 (time 1774528872 ; total 28 ; last 9) - stat 40000 (time 1774528880 ; total 36 ; last 8) - stat 50000 (time 1774528890 ; total 46 ; last 10) - stat 58973 (time 1774528901 ; total 57 ; last 11) - stat 60000 (time 1774528902 ; total 58 ; last 1) - stat 66938 (time 1774528913 ; total 69 ; last 11) - stat 70000 (time 1774528917 ; total 73 ; last 4) - stat 75971 (time 1774528928 ; total 84 ; last 11) - stat 80000 (time 1774528934 ; total 90 ; last 6) total: 87024 stats in 100 seconds: 870.239990 stats/second - unlinked 0 (time 1774528895 ; total 1 ; last 1) unlink(/mnt/lustre2/lockdir/lockfile4026) error: No such file or directory total: 4025 unlinks in 78 seconds: 51.602566 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19710) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19712) - No such process PASS 12 (134s) == sanityn test 13: test directory page revocation ======= 08:42:56 (1774528976) f13 f13 PASS 13 (5s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 08:43:02 (1774528982) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20881 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (5s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 08:43:06 (1774528986) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21453 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (4s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 08:43:10 (1774528990) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22025 Terminated $DIR1/$tdir/sleep 60 PASS 14b (4s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 08:43:14 (1774528994) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22601 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 08:43:18 (1774528998) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23176 Terminated $DIR1/$tdir/sleep 60 PASS 14d (5s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 08:43:23 (1774529003) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7527424 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7527424kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (17s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 08:43:40 (1774529020) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.377126 s, 5.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.309601 s, 6.8 MB/s Chance of close/open is 1 in 50 Seed set to 9023 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000004[1] 1774529024.239173 mapwrite 0x702df3 thru 0x71026b (0xd479 bytes) 000006[1] 1774529024.424241 write 0x86d0cc thru 0x87bd5d (0xec92 bytes) truncating to largest ever: 0x4de526 000007[1] 1774529024.433001 trunc from 0x87bd5e to 0x4de525 (0x39d838 bytes) 000008[0] 1774529024.461435 write 0x3643d4 thru 0x368214 (0x3e41 bytes) 000009[1] 1774529024.505482 mapwrite 0x516164 thru 0x51dfd1 (0x7e6e bytes) 000010[0] 1774529024.672900 mapread 0x00a90e thru 0x0183b4 (0xdaa7 bytes) 000012[0] 1774529024.774087 write 0x7ec148 thru 0x7f0e0e (0x4cc7 bytes) 000014[1] 1774529024.788121 read 0x0cfa3e thru 0x0d968b (0x9c4e bytes) 000015[0] 1774529024.802081 write 0x195bb6 thru 0x199cc5 (0x4110 bytes) 000018[0] 1774529024.850784 write 0x9dbe7d thru 0x9e9488 (0xd60c bytes) 000019[1] 1774529024.864122 mapread 0x1f5a27 thru 0x1f9f5b (0x4535 bytes) 000021[0] 1774529024.915153 write 0x6f2066 thru 0x6f55e0 (0x357b bytes) 000026[1] 1774529024.982869 write 0x34d15b thru 0x352a6a (0x5910 bytes) 000027[1] 1774529025.027176 mapwrite 0x0bf3f3 thru 0x0ca83a (0xb448 bytes) 000030[1] 1774529025.148885 mapwrite 0x94738f thru 0x952934 (0xb5a6 bytes) 000034[1] 1774529025.326717 read 0x5508bb thru 0x55706b (0x67b1 bytes) 000035[0] 1774529025.461875 read 0x010a0d thru 0x01ccba (0xc2ae bytes) 000038[1] 1774529025.482893 write 0x3093a5 thru 0x317112 (0xdd6e bytes) 000040[0] 1774529025.512052 write 0x0a8673 thru 0x0aaac1 (0x244f bytes) 000042[1] 1774529025.599023 read 0x21a8b1 thru 0x228e60 (0xe5b0 bytes) 000043[0] 1774529025.800906 mapwrite 0x68e4d2 thru 0x698b02 (0xa631 bytes) 000044[0] 1774529025.977120 write 0x14d5ca thru 0x159624 (0xc05b bytes) 000047[1] 1774529026.041347 write 0x7b2ee1 thru 0x7badae (0x7ece bytes) 000049[1] 1774529026.098142 write 0x584e71 thru 0x58bb96 (0x6d26 bytes) 000050[1] 1774529026.136379 mapread 0x3c8464 thru 0x3ce86c (0x6409 bytes) 000052[0] 1774529026.240345 mapwrite 0x3d3621 thru 0x3e222a (0xec0a bytes) 000054[0] 1774529026.369012 read 0x900419 thru 0x90fc80 (0xf868 bytes) 000055[1] 1774529026.394715 read 0x279aac thru 0x289792 (0xfce7 bytes) 000056[0] 1774529026.443071 write 0x56f75c thru 0x572bb9 (0x345e bytes) 000058[0] 1774529026.475067 mapwrite 0x731047 thru 0x73d7cf (0xc789 bytes) 000059[1] 1774529026.585839 write 0x7f7bf0 thru 0x80082c (0x8c3d bytes) 000060[1] 1774529026.776156 mapwrite 0x3cce19 thru 0x3d8d95 (0xbf7d bytes) 000061[1] 1774529026.915380 mapwrite 0x671449 thru 0x677bea (0x67a2 bytes) 000062[0] 1774529027.030053 read 0x9882d0 thru 0x993020 (0xad51 bytes) 000063[0] 1774529027.092870 read 0x09a9ac thru 0x09e703 (0x3d58 bytes) truncating to largest ever: 0x993c4d 000064[1] 1774529027.201967 trunc from 0x9e9489 to 0x993c4c (0x5583c bytes) 000065[0] 1774529027.345996 mapwrite 0x13fd6e thru 0x14e973 (0xec06 bytes) 000066[1] 1774529027.520763 mapwrite 0x6af0cf thru 0x6b0ca3 (0x1bd5 bytes) 000067[1] 1774529027.594172 trunc from 0x993c4d to 0x525f59 (0x46dcf3 bytes) 000068[1] 1774529027.636190 trunc from 0x525f5a to 0x4685d7 (0xbd982 bytes) 000071[0] 1774529027.675793 trunc from 0x4685d8 to 0x4bd356 (0x54d7f bytes) 000072[1] 1774529027.747753 trunc from 0x4bd357 to 0x5d21f3 (0x114e9d bytes) 000076[1] 1774529027.826310 mapwrite 0x394b0f thru 0x3a2fab (0xe49d bytes) 000079[0] 1774529027.986339 trunc from 0x5d21f4 to 0x939d55 (0x367b62 bytes) 000080[1] 1774529028.037340 trunc from 0x939d56 to 0x5b6f0f (0x382e46 bytes) 000081[0] 1774529028.082402 read 0x0e4224 thru 0x0e5485 (0x1262 bytes) 000083[1] 1774529028.097199 read 0x18ec3c thru 0x19041f (0x17e4 bytes) 000085[1] 1774529028.190190 mapwrite 0x9d684a thru 0x9da672 (0x3e29 bytes) 000086[0] 1774529028.409207 read 0x360c30 thru 0x36ceab (0xc27c bytes) 000087[1] 1774529028.466958 read 0x3f54fe thru 0x3fb366 (0x5e69 bytes) 000088[1] 1774529028.559622 trunc from 0x9da673 to 0x4f322a (0x4e7448 bytes) 000089[0] 1774529028.607711 mapwrite 0x4e656d thru 0x4f538f (0xee23 bytes) 000090[1] 1774529028.757073 read 0x3e1065 thru 0x3ee678 (0xd614 bytes) 000091[1] 1774529028.805384 mapwrite 0x251646 thru 0x257436 (0x5df1 bytes) 000092[0] 1774529028.940767 mapwrite 0x6faecb thru 0x6fddaa (0x2ee0 bytes) 000093[1] 1774529029.073197 mapwrite 0x3665cc thru 0x368fc4 (0x29f9 bytes) 000094[1] 1774529029.170507 trunc from 0x6fddab to 0x521a52 (0x1dc358 bytes) 000096[1] 1774529029.258504 write 0x408150 thru 0x414354 (0xc205 bytes) 000098[0] 1774529029.286282 mapread 0x2a62d6 thru 0x2b17bb (0xb4e6 bytes) 000100[1] 1774529029.382061 write 0x0c2793 thru 0x0c526c (0x2ada bytes) 000102[0] 1774529029.420794 read 0x036399 thru 0x039863 (0x34cb bytes) 000106[0] 1774529029.496946 mapread 0x1ced53 thru 0x1dca33 (0xdce1 bytes) 000107[0] 1774529029.603143 mapwrite 0x01a6e6 thru 0x028f9c (0xe8b7 bytes) 000108[1] 1774529029.749649 read 0x3a46cb thru 0x3ac54a (0x7e80 bytes) 000110[0] 1774529029.805454 read 0x16c0ef thru 0x16fa45 (0x3957 bytes) 000112[0] 1774529029.884711 write 0x1db2c9 thru 0x1e3159 (0x7e91 bytes) 000113[0] 1774529029.938359 mapread 0x500273 thru 0x50b0d7 (0xae65 bytes) 000115[0] 1774529030.011953 write 0x4ee9df thru 0x4f1272 (0x2894 bytes) 000116[1] 1774529030.013886 trunc from 0x521a53 to 0x8406c8 (0x31ec76 bytes) 000120[1] 1774529030.141768 read 0x4f09ae thru 0x4f3c65 (0x32b8 bytes) 000123[1] 1774529030.266782 mapread 0x4f9506 thru 0x4fb354 (0x1e4f bytes) 000124[0] 1774529030.267450 mapread 0x0d8fae thru 0x0e7608 (0xe65b bytes) 000126[1] 1774529030.356684 read 0x752d91 thru 0x75dba7 (0xae17 bytes) 000130[1] 1774529030.410703 mapread 0x305f9d thru 0x31219c (0xc200 bytes) 000131[1] 1774529030.487147 trunc from 0x8406c9 to 0x0823e9 (0x7be2df bytes) 000135[0] 1774529030.575329 mapwrite 0x2acdf0 thru 0x2b73dc (0xa5ed bytes) 000138[1] 1774529030.753468 write 0x2b4098 thru 0x2c23a8 (0xe311 bytes) 000139[0] 1774529030.804426 write 0x8172d5 thru 0x81fec5 (0x8bf1 bytes) 000141[0] 1774529030.872852 read 0x082b87 thru 0x08fbd7 (0xd051 bytes) 000143[1] 1774529030.985826 mapwrite 0x268a64 thru 0x276dc4 (0xe361 bytes) 000144[1] 1774529031.181637 trunc from 0x81fec6 to 0x3d4c66 (0x44b25f bytes) 000145[0] 1774529031.225441 mapwrite 0x088547 thru 0x08da09 (0x54c3 bytes) 000146[1] 1774529031.353007 write 0x57a192 thru 0x57df63 (0x3dd2 bytes) 000147[0] 1774529031.365953 read 0x4696f0 thru 0x46d2c6 (0x3bd7 bytes) 000149[0] 1774529031.447224 write 0x28d5aa thru 0x2950a7 (0x7afe bytes) 000150[0] 1774529031.484692 write 0x9457d8 thru 0x948c3a (0x3463 bytes) 000151[0] 1774529031.511741 write 0x554e7a thru 0x5649ef (0xfb76 bytes) 000152[1] 1774529031.586011 write 0x909fda thru 0x90b353 (0x137a bytes) 000153[0] 1774529031.635264 trunc from 0x948c3b to 0x837c27 (0x111013 bytes) 000154[1] 1774529031.689449 mapwrite 0x5b3eea thru 0x5c35bb (0xf6d2 bytes) 000156[0] 1774529031.858619 read 0x241669 thru 0x24e2e8 (0xcc80 bytes) 000157[1] 1774529031.894336 write 0x580184 thru 0x58d67f (0xd4fc bytes) 000158[1] 1774529031.915992 write 0x122d7f thru 0x124889 (0x1b0b bytes) 000160[1] 1774529031.950119 write 0x23a89a thru 0x248d67 (0xe4ce bytes) 000162[0] 1774529032.028031 mapwrite 0x738a02 thru 0x7447ef (0xbdee bytes) 000163[0] 1774529032.232564 write 0x42e1ae thru 0x42ec3c (0xa8f bytes) 000164[0] 1774529032.359103 read 0x36b1bc thru 0x3757a2 (0xa5e7 bytes) 000165[0] 1774529032.359776 mapwrite 0x136822 thru 0x14130a (0xaae9 bytes) 000166[0] 1774529032.502092 mapread 0x49435c thru 0x49c8cc (0x8571 bytes) 000168[1] 1774529032.533048 trunc from 0x837c28 to 0x5f3a5b (0x2441cc bytes) 000170[0] 1774529032.618994 mapread 0x41ceb0 thru 0x429450 (0xc5a1 bytes) 000171[1] 1774529032.711708 mapwrite 0x5fcde1 thru 0x606070 (0x9290 bytes) 000173[0] 1774529032.902923 write 0x4811ff thru 0x487f2c (0x6d2e bytes) 000179[1] 1774529033.111139 trunc from 0x606071 to 0x0401dd (0x5c5e93 bytes) 000180[0] 1774529033.187068 trunc from 0x0401de to 0x77839f (0x7381c2 bytes) 000181[0] 1774529033.238960 mapwrite 0x4750d0 thru 0x483a88 (0xe9b9 bytes) 000184[0] 1774529033.424029 mapread 0x58b880 thru 0x592d9e (0x751f bytes) 000185[1] 1774529033.463883 write 0x8f3e40 thru 0x8f656a (0x272b bytes) 000187[1] 1774529033.504085 read 0x63310d thru 0x642935 (0xf829 bytes) 000188[1] 1774529033.548358 trunc from 0x8f656b to 0x56926d (0x38d2fd bytes) 000189[0] 1774529033.621362 trunc from 0x56926e to 0x42f7a4 (0x139ac9 bytes) 000195[0] 1774529033.724394 mapwrite 0x83f5b7 thru 0x84af0b (0xb955 bytes) 000196[1] 1774529033.901232 write 0x7642c7 thru 0x772fe5 (0xed1f bytes) 000198[0] 1774529033.971847 mapread 0x281919 thru 0x28996e (0x8056 bytes) 000200[0] 1774529034.042145 mapwrite 0x43c9fc thru 0x4446dc (0x7ce1 bytes) 000201[0] 1774529034.141191 read 0x1bb4f9 thru 0x1c9574 (0xe07c bytes) 000202[0] 1774529034.273937 trunc from 0x84af0c to 0x2571a6 (0x5f3d65 bytes) 000203[1] 1774529034.325626 write 0x9005bd thru 0x90c232 (0xbc76 bytes) 000204[1] 1774529034.351238 mapread 0x0c207d thru 0x0c3d83 (0x1d07 bytes) 000205[1] 1774529034.439044 trunc from 0x90c233 to 0x199e96 (0x77239c bytes) 000206[1] 1774529034.532094 read 0x132446 thru 0x139f18 (0x7ad3 bytes) 000207[0] 1774529034.565509 mapread 0x064800 thru 0x06a8c8 (0x60c9 bytes) 000208[0] 1774529034.630042 trunc from 0x199e97 to 0x4339b2 (0x299b1c bytes) 000210[0] 1774529034.738571 read 0x1e254e thru 0x1e91e0 (0x6c93 bytes) 000213[1] 1774529034.827249 read 0x354b27 thru 0x3573de (0x28b8 bytes) 000216[0] 1774529034.878476 read 0x21c5be thru 0x2281fd (0xbc40 bytes) 000218[1] 1774529034.927320 mapread 0x09470a thru 0x0a3e26 (0xf71d bytes) 000219[0] 1774529035.070787 mapwrite 0x371db8 thru 0x37b878 (0x9ac1 bytes) 000220[0] 1774529035.247747 mapwrite 0x0bbcc1 thru 0x0c7220 (0xb560 bytes) 000222[1] 1774529035.391960 trunc from 0x4339b3 to 0x71a201 (0x2e684f bytes) 000224[0] 1774529035.513731 read 0x191973 thru 0x1992de (0x796c bytes) 000225[0] 1774529035.557969 read 0x5229e4 thru 0x5271ae (0x47cb bytes) 000226[0] 1774529035.592025 trunc from 0x71a202 to 0x5f2887 (0x12797a bytes) 000228[1] 1774529035.701459 trunc from 0x5f2888 to 0x7b9276 (0x1c69ef bytes) 000229[1] 1774529035.771928 read 0x6cb38f thru 0x6d615c (0xadce bytes) 000230[1] 1774529035.805773 mapread 0x4cf4b4 thru 0x4d04ee (0x103b bytes) 000231[1] 1774529035.836932 trunc from 0x7b9277 to 0x6aa418 (0x10ee5e bytes) 000233[1] 1774529035.897783 write 0x459f1d thru 0x45bf89 (0x206d bytes) 000234[1] 1774529035.910970 write 0x17bc3c thru 0x18bbde (0xffa3 bytes) 000235[1] 1774529035.943946 mapwrite 0x2e31de thru 0x2ed8b2 (0xa6d5 bytes) 000239[0] 1774529036.132927 trunc from 0x6aa419 to 0x35ca7d (0x34d99b bytes) 000240[1] 1774529036.211293 trunc from 0x35ca7e to 0x6b1468 (0x3549eb bytes) 000242[0] 1774529036.312747 write 0x95ff95 thru 0x96a3a2 (0xa40e bytes) 000244[0] 1774529036.400736 mapread 0x142312 thru 0x1505ec (0xe2db bytes) 000245[1] 1774529036.478784 trunc from 0x96a3a3 to 0x563543 (0x406e5f bytes) 000246[0] 1774529036.535153 mapwrite 0x7ab896 thru 0x7abadd (0x248 bytes) 000247[1] 1774529036.654961 write 0x006ea3 thru 0x0107e7 (0x9945 bytes) 000251[1] 1774529036.749984 mapread 0x36234e thru 0x3682ae (0x5f61 bytes) 000255[0] 1774529036.849340 read 0x3d7875 thru 0x3e2b0f (0xb29b bytes) 000256[1] 1774529036.880193 mapwrite 0x35498f thru 0x3593aa (0x4a1c bytes) 000258[1] 1774529036.982173 trunc from 0x7abade to 0x8fd251 (0x151774 bytes) 000260[0] 1774529037.005243 trunc from 0x8fd252 to 0x550166 (0x3ad0eb bytes) 000261[1] 1774529037.097078 trunc from 0x550167 to 0x842ed4 (0x2f2d6e bytes) 000263[0] 1774529037.212336 read 0x6463ce thru 0x6520ef (0xbd22 bytes) 000266[1] 1774529037.292795 write 0x6e88df thru 0x6f39ea (0xb10c bytes) 000271[1] 1774529037.415376 mapwrite 0x121829 thru 0x12ca79 (0xb251 bytes) 000274[1] 1774529037.571570 mapwrite 0x8cf0ac thru 0x8debab (0xfb00 bytes) 000275[0] 1774529037.643092 mapwrite 0x11e6b0 thru 0x12c9e3 (0xe334 bytes) 000277[1] 1774529037.969352 mapwrite 0x2b93e3 thru 0x2ba616 (0x1234 bytes) 000278[1] 1774529038.093018 mapread 0x640430 thru 0x643999 (0x356a bytes) 000279[0] 1774529038.154392 write 0x3092d1 thru 0x311cd4 (0x8a04 bytes) 000281[0] 1774529038.203169 mapread 0x05fbd4 thru 0x06be9b (0xc2c8 bytes) 000282[1] 1774529038.383408 mapread 0x167570 thru 0x16f5c7 (0x8058 bytes) 000285[0] 1774529038.462917 mapread 0x3a8c48 thru 0x3b08b9 (0x7c72 bytes) 000290[1] 1774529038.495588 read 0x3e914c thru 0x3f7587 (0xe43c bytes) 000292[0] 1774529038.567715 trunc from 0x8debac to 0x2f78b6 (0x5e72f5 bytes) 000293[0] 1774529038.648729 read 0x119558 thru 0x1217dd (0x8286 bytes) 000295[0] 1774529038.687035 trunc from 0x2f78b7 to 0x717b51 (0x42029b bytes) 000296[0] 1774529038.752333 read 0x4b20ea thru 0x4c1de6 (0xfcfd bytes) 000297[1] 1774529038.793196 mapwrite 0x22ad19 thru 0x23000d (0x52f5 bytes) 000298[0] 1774529038.927439 mapread 0x480a74 thru 0x487fd5 (0x7562 bytes) 000299[0] 1774529039.007669 trunc from 0x717b52 to 0x652144 (0xc5a0d bytes) 000301[0] 1774529039.069402 mapread 0x47f595 thru 0x48e36e (0xedda bytes) 000302[1] 1774529039.180165 read 0x1a46fa thru 0x1b2a84 (0xe38b bytes) 000307[0] 1774529039.357257 trunc from 0x652145 to 0x0c82e9 (0x589e5b bytes) 000310[1] 1774529039.448390 mapread 0x075b4f thru 0x078e6f (0x3321 bytes) 000311[1] 1774529039.520678 mapwrite 0x4d1222 thru 0x4debc3 (0xd9a2 bytes) 000312[1] 1774529039.717099 mapread 0x38b1b5 thru 0x38f965 (0x47b1 bytes) 000313[0] 1774529039.743459 write 0x025d14 thru 0x02d2b6 (0x75a3 bytes) 000315[1] 1774529039.824036 mapread 0x200ba4 thru 0x20f826 (0xec83 bytes) 000316[1] 1774529040.004083 write 0x59c632 thru 0x5a6ed5 (0xa8a4 bytes) 000317[0] 1774529040.007092 write 0x5c8c93 thru 0x5d0226 (0x7594 bytes) 000318[1] 1774529040.056305 mapwrite 0x7ebb72 thru 0x7ecfee (0x147d bytes) 000320[0] 1774529040.182875 write 0x1a12f9 thru 0x1aea52 (0xd75a bytes) 000322[1] 1774529040.249284 trunc from 0x7ecfef to 0x91f145 (0x132157 bytes) 000323[0] 1774529040.308929 mapwrite 0x3754d9 thru 0x37c11b (0x6c43 bytes) 000324[1] 1774529040.479581 trunc from 0x91f146 to 0x202e5e (0x71c2e7 bytes) 000325[0] 1774529040.545120 read 0x0e8775 thru 0x0eb413 (0x2c9f bytes) 000326[1] 1774529040.585113 write 0x52f4df thru 0x53b13a (0xbc5c bytes) 000328[0] 1774529040.590173 write 0x75b4d3 thru 0x760923 (0x5451 bytes) 000329[0] 1774529040.615269 read 0x4755d2 thru 0x480bc9 (0xb5f8 bytes) 000330[0] 1774529040.649080 mapwrite 0x2258b4 thru 0x22825f (0x29ac bytes) 000333[1] 1774529040.842013 write 0x60732f thru 0x60fed7 (0x8ba9 bytes) 000335[0] 1774529040.925429 read 0x3a6bdd thru 0x3acc89 (0x60ad bytes) 000336[0] 1774529040.953007 trunc from 0x760924 to 0x66b0e3 (0xf5840 bytes) 000337[0] 1774529040.993239 read 0x46c722 thru 0x477a89 (0xb368 bytes) 000338[1] 1774529041.077669 trunc from 0x66b0e4 to 0x7f94f8 (0x18e415 bytes) 000339[0] 1774529041.159041 trunc from 0x7f94f9 to 0x4f6c0e (0x3028ea bytes) 000340[0] 1774529041.207820 mapwrite 0x3a59d4 thru 0x3aff76 (0xa5a3 bytes) 000345[1] 1774529041.404923 write 0x45f33b thru 0x46ee9e (0xfb64 bytes) 000346[0] 1774529041.439972 trunc from 0x4f6c0f to 0x626245 (0x12f637 bytes) 000347[1] 1774529041.482757 write 0x6bcf2b thru 0x6c92b0 (0xc386 bytes) 000348[1] 1774529041.512950 mapread 0x22c4e5 thru 0x23b22c (0xed48 bytes) 000349[0] 1774529041.615805 mapread 0x4d3f8b thru 0x4de239 (0xa2af bytes) 000352[1] 1774529041.736018 trunc from 0x6c92b1 to 0x7249a0 (0x5b6f0 bytes) 000354[1] 1774529041.769637 trunc from 0x7249a1 to 0x390502 (0x39449e bytes) 000357[1] 1774529041.778972 mapwrite 0x4b5d94 thru 0x4b6f6d (0x11da bytes) 000360[0] 1774529041.863644 mapread 0x3e1f2b thru 0x3eda4c (0xbb22 bytes) 000362[0] 1774529041.921755 trunc from 0x4b6f6e to 0x30945f (0x1adb0e bytes) 000363[0] 1774529041.958475 trunc from 0x309460 to 0x22b3d8 (0xde087 bytes) 000364[1] 1774529041.985112 write 0x114df6 thru 0x11b08f (0x629a bytes) 000367[0] 1774529042.044045 read 0x177a2e thru 0x17d643 (0x5c16 bytes) 000368[1] 1774529042.129656 trunc from 0x22b3d9 to 0x60f062 (0x3e3c8a bytes) 000370[0] 1774529042.205599 read 0x2fb2f9 thru 0x308ea0 (0xdba8 bytes) 000371[0] 1774529042.324145 write 0x2b9ded thru 0x2c85ef (0xe803 bytes) 000373[0] 1774529042.358431 write 0x8e2f6d thru 0x8e8b0c (0x5ba0 bytes) 000375[1] 1774529042.360998 trunc from 0x8e8b0d to 0x7bce54 (0x12bcb8 bytes) 000376[0] 1774529042.426983 write 0x2c0e89 thru 0x2c220f (0x1387 bytes) 000377[0] 1774529042.498779 mapread 0x757963 thru 0x760531 (0x8bcf bytes) 000384[0] 1774529042.606001 trunc from 0x7bce55 to 0x4327a1 (0x38a6b3 bytes) 000385[1] 1774529042.668189 read 0x046a64 thru 0x0471d8 (0x775 bytes) 000386[1] 1774529042.722881 read 0x3bc5cb thru 0x3be9a6 (0x23dc bytes) 000388[0] 1774529042.755698 write 0x333d15 thru 0x33706f (0x335b bytes) 000390[0] 1774529042.801012 mapwrite 0x3a0745 thru 0x3a80b7 (0x7973 bytes) 000394[0] 1774529042.958167 mapwrite 0x211f92 thru 0x212c36 (0xca5 bytes) 000395[1] 1774529043.069595 write 0x2b566a thru 0x2c2255 (0xcbec bytes) 000397[1] 1774529043.144324 mapread 0x21f3da thru 0x228535 (0x915c bytes) 000399[0] 1774529043.251234 trunc from 0x4327a2 to 0x917ffb (0x4e585a bytes) 000402[0] 1774529043.342972 mapread 0x5ec8fe thru 0x5fb94a (0xf04d bytes) 000403[1] 1774529043.482932 mapwrite 0x955c08 thru 0x95fb30 (0x9f29 bytes) 000404[0] 1774529043.669458 mapread 0x03d4b9 thru 0x04304b (0x5b93 bytes) 000405[0] 1774529043.728654 mapread 0x04e000 thru 0x05c347 (0xe348 bytes) 000406[1] 1774529043.817481 trunc from 0x95fb31 to 0x5b30a5 (0x3aca8b bytes) 000407[0] 1774529043.883812 mapread 0x140394 thru 0x14d4ab (0xd118 bytes) 000408[1] 1774529043.982019 mapwrite 0x167f09 thru 0x16e866 (0x695e bytes) 000411[0] 1774529044.138166 read 0x2cd71c thru 0x2cf7ce (0x20b3 bytes) 000413[1] 1774529044.197799 read 0x2e532b thru 0x2f2be5 (0xd8bb bytes) 000415[0] 1774529044.287869 trunc from 0x5b30a6 to 0x1109db (0x4a26ca bytes) 000416[1] 1774529044.420621 mapread 0x0ca64a thru 0x0d3eb4 (0x986b bytes) 000417[0] 1774529044.546440 mapread 0x08479c thru 0x08b1e8 (0x6a4d bytes) 000418[0] 1774529044.614643 mapread 0x00401c thru 0x0094cd (0x54b2 bytes) 000419[1] 1774529044.672874 mapread 0x05355f thru 0x054687 (0x1129 bytes) 000420[1] 1774529044.693448 read 0x0d3a6d thru 0x0dca3b (0x8fcf bytes) 000422[0] 1774529044.724623 write 0x94212d thru 0x95115b (0xf02f bytes) 000423[1] 1774529044.752298 write 0x57bd8f thru 0x5861f4 (0xa466 bytes) 000425[0] 1774529044.982243 mapread 0x421f2f thru 0x42a5cb (0x869d bytes) 000426[0] 1774529045.188366 trunc from 0x95115c to 0x1ee316 (0x762e45 bytes) 000428[0] 1774529045.292894 mapwrite 0x6dcc8e thru 0x6e49b9 (0x7d2c bytes) 000432[0] 1774529045.536832 trunc from 0x6e49ba to 0x311ec2 (0x3d2af7 bytes) 000433[1] 1774529045.602529 write 0x08a35c thru 0x090b8a (0x682f bytes) 000434[0] 1774529045.641459 trunc from 0x311ec3 to 0x0776e7 (0x29a7db bytes) 000435[0] 1774529045.708158 trunc from 0x0776e8 to 0x92db97 (0x8b64b0 bytes) 000438[0] 1774529045.768587 mapread 0x71ccbd thru 0x71e937 (0x1c7b bytes) 000441[0] 1774529045.809079 mapwrite 0x5b4675 thru 0x5c0450 (0xbddc bytes) 000442[1] 1774529045.938214 write 0x439f80 thru 0x4433a0 (0x9421 bytes) 000443[0] 1774529045.987936 mapwrite 0x8ecec8 thru 0x8f004d (0x3186 bytes) 000444[1] 1774529046.109641 trunc from 0x92db98 to 0x7dfa17 (0x14e180 bytes) 000447[1] 1774529046.185146 write 0x1466f7 thru 0x153d3c (0xd646 bytes) 000448[0] 1774529046.198459 trunc from 0x7dfa18 to 0x2875bb (0x55845c bytes) 000449[0] 1774529046.263237 read 0x07a806 thru 0x0874fb (0xccf6 bytes) 000451[0] 1774529046.349928 mapread 0x1c1ab3 thru 0x1c6da5 (0x52f3 bytes) 000453[1] 1774529046.376450 write 0x16e212 thru 0x17376a (0x5559 bytes) 000455[1] 1774529046.435519 mapread 0x1e266d thru 0x1e28c4 (0x258 bytes) 000457[1] 1774529046.491195 read 0x2388e2 thru 0x246721 (0xde40 bytes) 000459[0] 1774529046.565081 read 0x21522e thru 0x21a0f2 (0x4ec5 bytes) 000461[0] 1774529046.603986 mapread 0x23a544 thru 0x24114c (0x6c09 bytes) 000463[0] 1774529046.638991 write 0x2fae6e thru 0x308881 (0xda14 bytes) 000464[1] 1774529046.725365 trunc from 0x308882 to 0x0b494a (0x253f37 bytes) 000465[1] 1774529046.838156 read 0x0a4319 thru 0x0a9804 (0x54ec bytes) 000467[1] 1774529046.858241 read 0x007619 thru 0x00d393 (0x5d7b bytes) 000468[1] 1774529046.880180 trunc from 0x0b494b to 0x20cc2e (0x1582e4 bytes) 000472[0] 1774529046.981616 trunc from 0x20cc2f to 0x768797 (0x55bb69 bytes) 000474[0] 1774529047.089417 trunc from 0x768798 to 0x2e8f03 (0x47f894 bytes) 000476[0] 1774529047.148814 mapread 0x2721c3 thru 0x278fe0 (0x6e1e bytes) 000477[1] 1774529047.237525 mapwrite 0x4797f5 thru 0x4823ec (0x8bf8 bytes) 000478[0] 1774529047.429495 read 0x08f42d thru 0x0965bf (0x7193 bytes) 000481[1] 1774529047.493411 read 0x23fe19 thru 0x24ae7b (0xb063 bytes) 000482[0] 1774529047.556448 trunc from 0x4823ed to 0x94db7b (0x4cb78f bytes) 000483[0] 1774529047.650102 write 0x7022ca thru 0x7082ed (0x6024 bytes) 000485[1] 1774529047.784154 write 0x1a4077 thru 0x1aa580 (0x650a bytes) 000486[0] 1774529047.852874 write 0x1062ac thru 0x110c9e (0xa9f3 bytes) 000487[0] 1774529047.889213 mapwrite 0x0704c5 thru 0x0757f0 (0x532c bytes) 000488[1] 1774529047.978231 trunc from 0x94db7c to 0x0bb239 (0x892942 bytes) 000489[1] 1774529048.080891 trunc from 0x0bb23a to 0x09d7bd (0x1da7c bytes) 000493[1] 1774529048.156458 mapwrite 0x7f394d thru 0x7f578f (0x1e43 bytes) 000494[0] 1774529048.347249 write 0x344278 thru 0x34deff (0x9c88 bytes) 000495[1] 1774529048.392697 mapwrite 0x2f6649 thru 0x2f98e9 (0x32a1 bytes) 000496[0] 1774529048.502529 trunc from 0x7f5790 to 0x346bbb (0x4aebd4 bytes) 000497[0] 1774529048.544104 trunc from 0x346bbc to 0x5b361f (0x26ca64 bytes) 000498[0] 1774529048.584260 read 0x592edf thru 0x5a25e3 (0xf705 bytes) 000500[1] 1774529048.599806 read 0x3564bd thru 0x35a64e (0x4192 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9048 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000003[0] 1774529049.224963 write_OD 0x5c6000 thru 0x5d4fff (0xf000 bytes) 000006[0] 1774529049.310115 mapread 0x41f000 thru 0x420d83 (0x1d84 bytes) truncating to largest ever: 0x12ed3b 000009[1] 1774529049.326352 trunc from 0x5d5000 to 0x12ed3a (0x4a62c5 bytes) 000010[0] 1774529049.396148 read_OD 0x001000 thru 0x00afff (0xa000 bytes) 000011[0] 1774529049.438400 read_OD 0x086000 thru 0x08dfff (0x8000 bytes) 000012[1] 1774529049.452317 write 0x390000 thru 0x394ad3 (0x4ad4 bytes) 000014[1] 1774529049.475273 write 0x36a000 thru 0x36bff6 (0x1ff7 bytes) 000016[1] 1774529049.484844 read 0x086000 thru 0x092f76 (0xcf77 bytes) 000017[1] 1774529049.495807 mapwrite 0x9a8000 thru 0x9b23ea (0xa3eb bytes) truncating to largest ever: 0x9e6c78 000018[0] 1774529049.655353 trunc from 0x9b23eb to 0x9e6c77 (0x3488d bytes) 000020[1] 1774529049.732073 trunc from 0x9e6c78 to 0x5f1235 (0x3f5a42 bytes) 000022[0] 1774529049.782429 read_OD 0x12a000 thru 0x134fff (0xb000 bytes) 000023[1] 1774529049.818832 mapread 0x5e1000 thru 0x5efa70 (0xea71 bytes) 000025[1] 1774529049.856729 mapwrite 0x742000 thru 0x74f93e (0xd93f bytes) 000026[0] 1774529050.045849 write_OD 0x142000 thru 0x150fff (0xf000 bytes) 000027[1] 1774529050.089825 mapread 0x022000 thru 0x0226ce (0x6cf bytes) 000031[1] 1774529050.103529 mapread 0x623000 thru 0x62c48c (0x948d bytes) 000032[0] 1774529050.126569 mapwrite 0x99b000 thru 0x9a9309 (0xe30a bytes) 000033[1] 1774529050.443206 write 0x376000 thru 0x3786d2 (0x26d3 bytes) 000035[1] 1774529050.490213 mapwrite 0x891000 thru 0x89dd24 (0xcd25 bytes) 000036[0] 1774529050.640998 read_OD 0x29b000 thru 0x29cfff (0x2000 bytes) 000037[0] 1774529050.699294 mapread 0x6d0000 thru 0x6da92d (0xa92e bytes) 000038[1] 1774529050.809840 mapwrite 0x7f9000 thru 0x7fb88c (0x288d bytes) 000039[1] 1774529050.939149 mapwrite 0x3d0000 thru 0x3dba59 (0xba5a bytes) 000040[0] 1774529051.048388 write_OD 0x1b7000 thru 0x1c3fff (0xd000 bytes) 000041[1] 1774529051.128681 read 0x26b000 thru 0x27604e (0xb04f bytes) 000042[0] 1774529051.235172 mapread 0x593000 thru 0x59563a (0x263b bytes) 000043[1] 1774529051.306095 read 0x540000 thru 0x54c5dc (0xc5dd bytes) 000045[0] 1774529051.306971 read_OD 0x578000 thru 0x57bfff (0x4000 bytes) 000048[1] 1774529051.326017 read 0x6bf000 thru 0x6cc02a (0xd02b bytes) 000057[1] 1774529051.329090 mapread 0x576000 thru 0x57cd4b (0x6d4c bytes) 000059[0] 1774529051.330812 mapwrite 0x20f000 thru 0x21b6cc (0xc6cd bytes) 000060[1] 1774529051.693807 mapread 0x823000 thru 0x832289 (0xf28a bytes) 000061[0] 1774529051.792117 mapwrite 0x4c3000 thru 0x4c98ae (0x68af bytes) 000062[1] 1774529051.869250 trunc from 0x9a930a to 0x817c02 (0x191707 bytes) 000063[0] 1774529051.897590 mapread 0x6e4000 thru 0x6edc21 (0x9c22 bytes) 000064[0] 1774529051.960366 write_OD 0x1a3000 thru 0x1acfff (0xa000 bytes) 000065[1] 1774529052.027010 read 0x5ea000 thru 0x5f3d42 (0x9d43 bytes) 000066[1] 1774529052.077461 mapread 0x201000 thru 0x20de76 (0xce77 bytes) 000068[1] 1774529052.127654 trunc from 0x817c03 to 0x7f0186 (0x27a7c bytes) 000069[0] 1774529052.154852 write_OD 0x650000 thru 0x65afff (0xb000 bytes) 000073[0] 1774529052.212173 mapwrite 0x4d3000 thru 0x4e2c40 (0xfc41 bytes) 000075[0] 1774529052.329730 trunc from 0x7f0187 to 0x5fade8 (0x1f539e bytes) 000077[1] 1774529052.372810 mapread 0x0fb000 thru 0x109cd8 (0xecd9 bytes) 000080[1] 1774529052.450029 mapread 0x063000 thru 0x0715e1 (0xe5e2 bytes) 000081[1] 1774529052.514396 read 0x5e2000 thru 0x5e8a7f (0x6a80 bytes) 000084[1] 1774529052.524715 read 0x0cd000 thru 0x0d8713 (0xb714 bytes) 000085[0] 1774529052.588234 read_OD 0x0fd000 thru 0x109fff (0xd000 bytes) 000087[1] 1774529052.615609 mapread 0x051000 thru 0x05784d (0x684e bytes) 000089[1] 1774529052.667638 read 0x313000 thru 0x3214b2 (0xe4b3 bytes) 000090[1] 1774529052.668075 write 0x85d000 thru 0x869694 (0xc695 bytes) 000091[1] 1774529052.690326 read 0x330000 thru 0x33255e (0x255f bytes) 000094[0] 1774529052.850519 mapread 0x574000 thru 0x57d72c (0x972d bytes) 000095[0] 1774529053.012059 read_OD 0x06e000 thru 0x07bfff (0xe000 bytes) 000098[1] 1774529053.039200 mapwrite 0x9de000 thru 0x9e7a7e (0x9a7f bytes) 000100[0] 1774529053.226895 mapwrite 0x935000 thru 0x93d10e (0x810f bytes) 000104[0] 1774529053.471965 mapwrite 0x98f000 thru 0x99b557 (0xc558 bytes) 000106[1] 1774529053.549014 mapread 0x5aa000 thru 0x5b136f (0x7370 bytes) 000108[1] 1774529053.616993 mapread 0x803000 thru 0x809f70 (0x6f71 bytes) 000109[1] 1774529053.641484 mapread 0x58a000 thru 0x58c429 (0x242a bytes) 000111[0] 1774529053.652180 read_OD 0x00e000 thru 0x011fff (0x4000 bytes) 000112[1] 1774529053.657160 write 0x71a000 thru 0x721766 (0x7767 bytes) 000116[1] 1774529053.683315 write 0x279000 thru 0x27abb1 (0x1bb2 bytes) 000117[0] 1774529053.688861 trunc from 0x9e7a7f to 0x4d6642 (0x51143c bytes) 000118[0] 1774529053.731214 read_OD 0x38a000 thru 0x395fff (0xc000 bytes) 000120[1] 1774529053.753658 write 0x4ff000 thru 0x4ff257 (0x258 bytes) 000121[0] 1774529053.772303 trunc from 0x4ff258 to 0x356494 (0x1a8dc3 bytes) 000122[1] 1774529053.844995 write 0x0b9000 thru 0x0c68e8 (0xd8e9 bytes) 000126[0] 1774529053.897767 write_OD 0x7da000 thru 0x7dafff (0x1000 bytes) 000127[0] 1774529053.972744 mapwrite 0x8e9000 thru 0x8f217e (0x917f bytes) 000129[0] 1774529054.219052 mapread 0x1ec000 thru 0x1f6732 (0xa733 bytes) 000132[1] 1774529054.355751 write 0x620000 thru 0x625ba9 (0x5baa bytes) 000134[0] 1774529054.402183 read_OD 0x47b000 thru 0x484fff (0xa000 bytes) 000135[0] 1774529054.474167 mapread 0x8e3000 thru 0x8e72a5 (0x42a6 bytes) 000136[1] 1774529054.556889 mapwrite 0x37f000 thru 0x38e741 (0xf742 bytes) 000137[1] 1774529054.725193 mapread 0x891000 thru 0x89f89f (0xe8a0 bytes) 000140[1] 1774529054.797202 read 0x22c000 thru 0x239600 (0xd601 bytes) 000142[0] 1774529054.926488 mapread 0x8ba000 thru 0x8bb45a (0x145b bytes) 000145[0] 1774529054.968317 read_OD 0x599000 thru 0x59efff (0x6000 bytes) 000146[0] 1774529054.980703 trunc from 0x8f217f to 0x6848a4 (0x26d8da bytes) 000148[1] 1774529055.022240 read 0x568000 thru 0x56ce95 (0x4e96 bytes) 000150[1] 1774529055.056717 read 0x5ba000 thru 0x5c899a (0xe99b bytes) 000155[0] 1774529055.057930 mapread 0x64f000 thru 0x65edd7 (0xfdd8 bytes) 000156[1] 1774529055.082113 trunc from 0x6848a5 to 0x52069b (0x164209 bytes) 000158[0] 1774529055.197256 write_OD 0x9f6000 thru 0x9fefff (0x9000 bytes) 000159[1] 1774529055.285084 write 0x1e9000 thru 0x1f0a40 (0x7a41 bytes) 000161[1] 1774529055.305874 mapread 0x86b000 thru 0x87773e (0xc73f bytes) 000162[0] 1774529055.394512 mapwrite 0x70a000 thru 0x70e985 (0x4986 bytes) 000164[0] 1774529055.562156 mapwrite 0x327000 thru 0x32c9ad (0x59ae bytes) 000165[1] 1774529055.645995 write 0x514000 thru 0x51c677 (0x8678 bytes) 000167[1] 1774529055.700869 mapread 0x8b0000 thru 0x8b28c5 (0x28c6 bytes) 000169[0] 1774529055.744222 trunc from 0x9ff000 to 0x13d48e (0x8c1b71 bytes) 000170[1] 1774529055.793190 mapwrite 0x56f000 thru 0x572f70 (0x3f71 bytes) 000173[1] 1774529055.979112 write 0x483000 thru 0x4860e4 (0x30e5 bytes) 000174[0] 1774529056.022493 mapread 0x082000 thru 0x0865c1 (0x45c2 bytes) 000175[0] 1774529056.097700 mapread 0x386000 thru 0x38d218 (0x7219 bytes) 000177[1] 1774529056.137702 write 0x3b2000 thru 0x3bb1f0 (0x91f1 bytes) 000179[0] 1774529056.179996 mapread 0x42c000 thru 0x43b550 (0xf551 bytes) 000180[0] 1774529056.301528 trunc from 0x572f71 to 0x11bf0f (0x457061 bytes) 000182[1] 1774529056.365150 trunc from 0x11bf10 to 0x3e4870 (0x2c8961 bytes) 000184[0] 1774529056.434971 write_OD 0x670000 thru 0x67dfff (0xe000 bytes) 000185[0] 1774529056.515045 trunc from 0x67e000 to 0x0861df (0x5f7e20 bytes) 000187[1] 1774529056.574909 mapread 0x05f000 thru 0x0631d3 (0x41d4 bytes) 000188[1] 1774529056.631780 mapread 0x035000 thru 0x038dbf (0x3dc0 bytes) 000190[0] 1774529056.653854 trunc from 0x0861e0 to 0x142860 (0xbc681 bytes) 000193[1] 1774529056.775873 mapwrite 0x108000 thru 0x11576a (0xd76b bytes) 000195[1] 1774529056.946532 mapwrite 0x034000 thru 0x042f48 (0xef49 bytes) 000196[1] 1774529057.105426 mapread 0x043000 thru 0x043fd4 (0xfd5 bytes) 000197[1] 1774529057.134187 mapwrite 0x294000 thru 0x2a3095 (0xf096 bytes) 000198[1] 1774529057.313088 trunc from 0x2a3096 to 0x983b48 (0x6e0ab3 bytes) 000200[1] 1774529057.374219 mapwrite 0x758000 thru 0x7618e1 (0x98e2 bytes) 000204[1] 1774529057.551811 read 0x78d000 thru 0x796ddf (0x9de0 bytes) 000206[0] 1774529057.686532 mapread 0x609000 thru 0x61606b (0xd06c bytes) 000207[0] 1774529057.796660 read_OD 0x64f000 thru 0x65dfff (0xf000 bytes) 000210[0] 1774529057.806521 read_OD 0x31e000 thru 0x32cfff (0xf000 bytes) 000211[1] 1774529057.814874 write 0x04c000 thru 0x04d765 (0x1766 bytes) 000214[0] 1774529057.848103 mapwrite 0x65a000 thru 0x666c57 (0xcc58 bytes) 000216[0] 1774529058.161202 trunc from 0x983b49 to 0x810ec5 (0x172c83 bytes) 000217[0] 1774529058.236160 read_OD 0x4a1000 thru 0x4a9fff (0x9000 bytes) 000218[0] 1774529058.282951 mapread 0x3dd000 thru 0x3e66e0 (0x96e1 bytes) 000219[1] 1774529058.447433 trunc from 0x810ec6 to 0x9a7eaa (0x196fe5 bytes) 000221[1] 1774529058.501073 mapread 0x4a9000 thru 0x4ae328 (0x5329 bytes) 000222[0] 1774529058.570202 trunc from 0x9a7eab to 0x0daf5f (0x8ccf4b bytes) 000224[1] 1774529058.628861 mapwrite 0x9cf000 thru 0x9deeb3 (0xfeb4 bytes) 000229[0] 1774529058.792918 write_OD 0x6e4000 thru 0x6f1fff (0xe000 bytes) 000231[1] 1774529058.856370 mapwrite 0x3b2000 thru 0x3b4f29 (0x2f2a bytes) 000232[0] 1774529058.977756 mapread 0x771000 thru 0x772c80 (0x1c81 bytes) 000233[0] 1774529059.084288 mapwrite 0x576000 thru 0x578697 (0x2698 bytes) 000234[1] 1774529059.167911 trunc from 0x9deeb4 to 0x71d627 (0x2c188c bytes) 000235[1] 1774529059.206820 trunc from 0x71d628 to 0x9b0b11 (0x2934ea bytes) 000242[1] 1774529059.266487 trunc from 0x9b0b12 to 0x778b59 (0x237fb8 bytes) 000243[1] 1774529059.289867 read 0x1fc000 thru 0x20623f (0xa240 bytes) 000244[1] 1774529059.377486 read 0x63d000 thru 0x63f403 (0x2404 bytes) 000246[1] 1774529059.432656 write 0x002000 thru 0x010213 (0xe214 bytes) 000249[0] 1774529059.510513 write_OD 0x5bc000 thru 0x5cafff (0xf000 bytes) 000252[0] 1774529059.656886 trunc from 0x778b5a to 0x9304c0 (0x1b7967 bytes) 000253[1] 1774529059.706421 mapwrite 0x826000 thru 0x827a1c (0x1a1d bytes) 000254[1] 1774529059.794060 mapwrite 0x6a0000 thru 0x6a9ed2 (0x9ed3 bytes) 000256[0] 1774529059.923458 trunc from 0x9304c1 to 0x6052e3 (0x32b1dd bytes) 000257[0] 1774529059.955996 mapwrite 0x696000 thru 0x6a386b (0xd86c bytes) 000261[1] 1774529060.179975 mapwrite 0x0cf000 thru 0x0d1df1 (0x2df2 bytes) 000262[0] 1774529060.279389 write_OD 0x34b000 thru 0x34bfff (0x1000 bytes) 000265[1] 1774529060.349991 mapwrite 0x466000 thru 0x4731cd (0xd1ce bytes) 000269[0] 1774529060.559623 trunc from 0x6a386c to 0x7a2d8b (0xff520 bytes) 000271[0] 1774529060.605691 mapread 0x5c2000 thru 0x5d175c (0xf75d bytes) 000272[1] 1774529060.692903 read 0x0d8000 thru 0x0ded3a (0x6d3b bytes) 000273[0] 1774529060.852316 read_OD 0x2a6000 thru 0x2b2fff (0xd000 bytes) 000274[0] 1774529060.881994 write_OD 0x68d000 thru 0x693fff (0x7000 bytes) 000279[1] 1774529060.975053 read 0x70b000 thru 0x714668 (0x9669 bytes) 000280[1] 1774529060.992780 trunc from 0x7a2d8c to 0x9a3338 (0x2005ad bytes) 000283[0] 1774529061.058338 trunc from 0x9a3339 to 0x240d87 (0x7625b1 bytes) 000287[0] 1774529061.150119 mapread 0x043000 thru 0x04586b (0x286c bytes) 000288[1] 1774529061.198274 read 0x1b4000 thru 0x1bda8a (0x9a8b bytes) 000289[1] 1774529061.229732 write 0x489000 thru 0x489809 (0x80a bytes) 000290[0] 1774529061.238377 trunc from 0x48980a to 0x15017e (0x33968b bytes) 000291[1] 1774529061.303235 mapread 0x020000 thru 0x02ab93 (0xab94 bytes) 000292[0] 1774529061.396446 mapread 0x01a000 thru 0x02244e (0x844f bytes) 000294[0] 1774529061.462883 write_OD 0x61b000 thru 0x626fff (0xc000 bytes) 000297[0] 1774529061.560903 mapwrite 0x27d000 thru 0x28cb7e (0xfb7f bytes) 000298[1] 1774529061.754764 trunc from 0x627000 to 0x89838e (0x27138f bytes) 000299[0] 1774529061.829054 mapread 0x52f000 thru 0x537ddc (0x8ddd bytes) 000300[0] 1774529061.960845 mapread 0x018000 thru 0x021f99 (0x9f9a bytes) 000302[1] 1774529062.020009 write 0x1fe000 thru 0x2082fc (0xa2fd bytes) 000304[0] 1774529062.248863 read_OD 0x39d000 thru 0x3aafff (0xe000 bytes) 000305[0] 1774529062.337766 mapread 0x044000 thru 0x052f8b (0xef8c bytes) 000308[1] 1774529062.618024 mapread 0x2f2000 thru 0x2ffb24 (0xdb25 bytes) 000310[1] 1774529062.716875 read 0x057000 thru 0x05d4a0 (0x64a1 bytes) 000311[0] 1774529062.855800 mapwrite 0x394000 thru 0x3979b8 (0x39b9 bytes) 000312[0] 1774529062.979119 read_OD 0x39d000 thru 0x3a6fff (0xa000 bytes) 000313[1] 1774529063.017171 write 0x74c000 thru 0x7554f9 (0x94fa bytes) 000314[0] 1774529063.041505 mapread 0x557000 thru 0x565d0f (0xed10 bytes) 000315[0] 1774529063.167714 write_OD 0x812000 thru 0x813fff (0x2000 bytes) 000316[0] 1774529063.265261 mapread 0x499000 thru 0x4a0bab (0x7bac bytes) 000317[0] 1774529063.327894 mapread 0x3d2000 thru 0x3dcdbe (0xadbf bytes) 000318[0] 1774529063.396566 write_OD 0x152000 thru 0x152fff (0x1000 bytes) 000322[1] 1774529063.487354 read 0x17c000 thru 0x18255c (0x655d bytes) 000323[0] 1774529063.616412 read_OD 0x88c000 thru 0x895fff (0xa000 bytes) 000324[0] 1774529063.632086 trunc from 0x89838f to 0x5b54f5 (0x2e2e99 bytes) 000328[0] 1774529063.813808 trunc from 0x5b54f6 to 0x9490a0 (0x393bab bytes) 000329[0] 1774529063.881377 trunc from 0x9490a1 to 0x6c7241 (0x281e5f bytes) 000331[1] 1774529063.918709 mapwrite 0x3a9000 thru 0x3b56bb (0xc6bc bytes) 000333[1] 1774529064.069607 write 0x524000 thru 0x5310ac (0xd0ad bytes) 000336[0] 1774529064.082732 read_OD 0x382000 thru 0x38bfff (0xa000 bytes) 000339[1] 1774529064.109656 mapread 0x65b000 thru 0x665bdd (0xabde bytes) 000341[0] 1774529064.155855 mapwrite 0x71b000 thru 0x724c8f (0x9c90 bytes) 000342[0] 1774529064.305766 mapwrite 0x52a000 thru 0x52af99 (0xf9a bytes) 000343[0] 1774529064.372885 mapread 0x6e7000 thru 0x6f5f64 (0xef65 bytes) 000345[1] 1774529064.410876 trunc from 0x724c90 to 0x9ca030 (0x2a53a1 bytes) 000347[0] 1774529064.501946 trunc from 0x9ca031 to 0x7f1bf3 (0x1d843d bytes) 000348[1] 1774529064.556325 trunc from 0x7f1bf4 to 0x388349 (0x4698aa bytes) 000353[1] 1774529064.652239 trunc from 0x38834a to 0x4194d1 (0x91188 bytes) 000354[0] 1774529064.688772 mapwrite 0x6f9000 thru 0x70158b (0x858c bytes) 000355[1] 1774529064.806926 read 0x34e000 thru 0x3564fe (0x84ff bytes) 000358[0] 1774529064.908255 mapwrite 0x247000 thru 0x2545d9 (0xd5da bytes) 000359[0] 1774529065.104334 read_OD 0x436000 thru 0x443fff (0xe000 bytes) 000366[0] 1774529065.134909 mapread 0x1cd000 thru 0x1db4a2 (0xe4a3 bytes) 000367[0] 1774529065.293492 write_OD 0x708000 thru 0x70dfff (0x6000 bytes) 000369[1] 1774529065.367646 mapwrite 0x639000 thru 0x648889 (0xf88a bytes) 000370[0] 1774529065.489939 write_OD 0x78a000 thru 0x791fff (0x8000 bytes) 000371[0] 1774529065.557870 write_OD 0x528000 thru 0x535fff (0xe000 bytes) 000372[1] 1774529065.621508 mapread 0x129000 thru 0x134acc (0xbacd bytes) 000373[0] 1774529065.715128 write_OD 0x607000 thru 0x60bfff (0x5000 bytes) 000374[1] 1774529065.795178 trunc from 0x792000 to 0x4886b3 (0x30994c bytes) 000376[0] 1774529065.839761 trunc from 0x4886b4 to 0x1b3fc7 (0x2d46ec bytes) 000377[1] 1774529065.885800 mapwrite 0x528000 thru 0x536ce9 (0xecea bytes) 000379[1] 1774529066.067111 read 0x384000 thru 0x38a8cf (0x68d0 bytes) 000382[0] 1774529066.139119 read_OD 0x065000 thru 0x06ffff (0xb000 bytes) 000384[1] 1774529066.166286 write 0x99c000 thru 0x9ab19e (0xf19f bytes) 000385[0] 1774529066.178326 write_OD 0x927000 thru 0x930fff (0xa000 bytes) 000386[0] 1774529066.233149 read_OD 0x3bf000 thru 0x3cafff (0xc000 bytes) 000388[1] 1774529066.243117 trunc from 0x9ab19f to 0x79930d (0x211e91 bytes) 000390[0] 1774529066.293948 mapwrite 0x3aa000 thru 0x3ade22 (0x3e23 bytes) 000391[1] 1774529066.442831 mapwrite 0x06c000 thru 0x0774f9 (0xb4fa bytes) 000393[0] 1774529066.632908 trunc from 0x79930e to 0x428671 (0x370c9c bytes) 000394[0] 1774529066.674115 write_OD 0x34e000 thru 0x35bfff (0xe000 bytes) 000395[1] 1774529066.720140 write 0x0bb000 thru 0x0c8ab8 (0xdab9 bytes) 000396[0] 1774529066.755926 trunc from 0x428672 to 0x9bef48 (0x5968d7 bytes) 000399[0] 1774529066.810438 mapread 0x4eb000 thru 0x4f96ee (0xe6ef bytes) 000402[1] 1774529066.922019 trunc from 0x9bef49 to 0x7460dd (0x278e6b bytes) 000405[0] 1774529066.987103 write_OD 0x012000 thru 0x01efff (0xd000 bytes) 000410[1] 1774529067.051933 write 0x6dc000 thru 0x6e2fef (0x6ff0 bytes) 000413[0] 1774529067.122796 write_OD 0x3f3000 thru 0x3fdfff (0xb000 bytes) 000418[0] 1774529067.175204 mapwrite 0x8e0000 thru 0x8e485e (0x485f bytes) 000419[0] 1774529067.265749 trunc from 0x8e485f to 0x1453bf (0x79f49f bytes) 000421[1] 1774529067.281081 read 0x0fe000 thru 0x1033d9 (0x53da bytes) 000422[0] 1774529067.311511 mapread 0x0a0000 thru 0x0affcc (0xffcd bytes) 000423[1] 1774529067.379041 mapread 0x0bc000 thru 0x0c53cd (0x93ce bytes) 000424[0] 1774529067.417860 mapread 0x071000 thru 0x07fdc9 (0xedca bytes) 000425[1] 1774529067.477440 trunc from 0x1453c0 to 0x06f956 (0xd5a69 bytes) 000432[0] 1774529067.594304 write_OD 0x0d9000 thru 0x0dcfff (0x4000 bytes) 000433[1] 1774529067.641135 mapread 0x021000 thru 0x02dc8d (0xcc8e bytes) 000436[1] 1774529067.703822 read 0x01c000 thru 0x025101 (0x9102 bytes) 000437[0] 1774529067.722451 mapwrite 0x7ce000 thru 0x7d5b94 (0x7b95 bytes) 000438[0] 1774529067.843796 trunc from 0x7d5b95 to 0x442a2f (0x393165 bytes) 000440[0] 1774529067.870622 mapwrite 0x50f000 thru 0x51dc83 (0xec84 bytes) 000442[0] 1774529068.058743 mapwrite 0x91e000 thru 0x927440 (0x9441 bytes) 000444[1] 1774529068.224637 trunc from 0x927441 to 0x090ca9 (0x896797 bytes) 000445[1] 1774529068.246526 read 0x033000 thru 0x042d10 (0xfd11 bytes) 000446[0] 1774529068.286463 trunc from 0x090caa to 0x59320c (0x502563 bytes) 000447[0] 1774529068.367614 read_OD 0x290000 thru 0x292fff (0x3000 bytes) 000448[1] 1774529068.411961 read 0x2d9000 thru 0x2e1a0f (0x8a10 bytes) 000450[0] 1774529068.514754 write_OD 0x3da000 thru 0x3e0fff (0x7000 bytes) 000451[0] 1774529068.594975 trunc from 0x59320d to 0x34da5b (0x2457b1 bytes) 000452[0] 1774529068.645599 read_OD 0x2eb000 thru 0x2f0fff (0x6000 bytes) 000454[1] 1774529068.680949 read 0x045000 thru 0x04ae92 (0x5e93 bytes) 000456[1] 1774529068.755390 write 0x994000 thru 0x999e48 (0x5e49 bytes) 000459[0] 1774529068.842045 write_OD 0x61a000 thru 0x61efff (0x5000 bytes) 000462[1] 1774529068.928970 write 0x5bb000 thru 0x5c8189 (0xd18a bytes) 000463[0] 1774529069.005482 trunc from 0x999e49 to 0x97c0aa (0x1dd9e bytes) 000465[1] 1774529069.071798 read 0x14b000 thru 0x1518c3 (0x68c4 bytes) 000466[0] 1774529069.243616 mapwrite 0x030000 thru 0x03604d (0x604e bytes) 000467[0] 1774529069.577979 mapread 0x8e5000 thru 0x8f3250 (0xe251 bytes) 000469[1] 1774529069.579052 mapread 0x240000 thru 0x24e104 (0xe105 bytes) 000470[0] 1774529069.826144 read_OD 0x6d9000 thru 0x6e0fff (0x8000 bytes) truncating to largest ever: 0x9f73a8 000475[0] 1774529069.849776 trunc from 0x97c0ab to 0x9f73a7 (0x7b2fd bytes) 000476[0] 1774529069.905073 trunc from 0x9f73a8 to 0x0563a2 (0x9a1005 bytes) 000478[1] 1774529069.991334 read 0x001000 thru 0x00db45 (0xcb46 bytes) 000479[0] 1774529070.029382 mapwrite 0x383000 thru 0x38475f (0x1760 bytes) 000482[1] 1774529070.157095 mapwrite 0x399000 thru 0x3a0f49 (0x7f4a bytes) 000483[1] 1774529070.324331 mapwrite 0x95e000 thru 0x95f9f0 (0x19f1 bytes) 000486[1] 1774529070.407248 trunc from 0x95f9f1 to 0x869e3d (0xf5bb3 bytes) 000488[1] 1774529070.424301 write 0x92d000 thru 0x930885 (0x3886 bytes) 000491[0] 1774529070.460895 write_OD 0x035000 thru 0x037fff (0x3000 bytes) 000493[0] 1774529070.506080 mapwrite 0x279000 thru 0x287679 (0xe67a bytes) 000495[1] 1774529070.855487 trunc from 0x930886 to 0x239862 (0x6f7023 bytes) 000496[0] 1774529070.914673 read_OD 0x21e000 thru 0x229fff (0xc000 bytes) 000497[0] 1774529070.972177 read_OD 0x0ba000 thru 0x0c0fff (0x7000 bytes) 000499[0] 1774529070.999433 write_OD 0x594000 thru 0x59afff (0x7000 bytes) 000500[1] 1774529071.069216 mapread 0x57d000 thru 0x5854d0 (0x84d1 bytes) All operations completed A-OK! PASS 16a (55s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 08:44:35 (1774529075) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.366081 s, 5.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.179194 s, 11.7 MB/s Chance of close/open is 1 in 50 Seed set to 9077 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x554f27 000001[1] 1774529078.133848 trunc from 00000000 to 0x554f26 (0x554f27 bytes) 000009[0] 1774529078.201189 trunc from 0x554f27 to 0x2f628b (0x25ec9b bytes) 000010[1] 1774529078.243815 read 0x059809 thru 0x059a36 (0x22e bytes) 000013[0] 1774529078.293999 write 0x55b5a6 thru 0x55ca89 (0x14e4 bytes) 000015[1] 1774529078.296125 trunc from 0x55ca8a to 0x20db7d (0x34ef0c bytes) 000016[0] 1774529078.340240 read 0x13814b thru 0x139e33 (0x1ce9 bytes) 000019[0] 1774529078.386196 mapread 0x0c2658 thru 0x0c3746 (0x10ef bytes) 000022[0] 1774529078.428242 mapread 0x09a4b8 thru 0x09c44a (0x1f93 bytes) 000024[1] 1774529078.446674 mapread 0x09185a thru 0x09208f (0x836 bytes) 000027[0] 1774529078.460704 mapread 0x0a1e05 thru 0x0a3a6c (0x1c68 bytes) 000028[0] 1774529078.472846 write 0x2f7c48 thru 0x2f83e9 (0x7a2 bytes) 000029[1] 1774529078.516379 write 0x8098dd thru 0x80b72f (0x1e53 bytes) 000030[1] 1774529078.582138 trunc from 0x80b730 to 0x29f946 (0x56bde9 bytes) 000033[1] 1774529078.663817 write 0x2ab80a thru 0x2abbf7 (0x3ee bytes) 000034[0] 1774529078.665511 read 0x0f7a7c thru 0x0f8766 (0xceb bytes) 000035[0] 1774529078.703854 mapread 0x0df821 thru 0x0e0868 (0x1048 bytes) 000037[0] 1774529078.745476 mapwrite 0x055ccb thru 0x057af7 (0x1e2d bytes) 000039[0] 1774529078.898659 write 0x0ca382 thru 0x0ca6eb (0x36a bytes) truncating to largest ever: 0x829a2b 000041[1] 1774529078.939043 trunc from 0x2abbf8 to 0x829a2a (0x57de33 bytes) 000045[0] 1774529079.038237 write 0x77191a thru 0x772add (0x11c4 bytes) 000046[1] 1774529079.109994 mapwrite 0x815a9e thru 0x81779c (0x1cff bytes) 000047[1] 1774529079.214849 mapread 0x09ca5e thru 0x09cc5f (0x202 bytes) 000048[1] 1774529079.244810 write 0x6aa4e7 thru 0x6ab8c4 (0x13de bytes) 000049[1] 1774529079.279161 write 0x85b2a5 thru 0x85c27e (0xfda bytes) 000050[0] 1774529079.300394 mapwrite 0x20b5b3 thru 0x20c36e (0xdbc bytes) 000051[0] 1774529079.475533 read 0x0cee0d thru 0x0cf5a0 (0x794 bytes) 000053[1] 1774529079.490471 mapread 0x3d49a8 thru 0x3d4f54 (0x5ad bytes) 000054[0] 1774529079.559972 write 0x0fd0af thru 0x0fe0b6 (0x1008 bytes) 000055[0] 1774529079.588306 mapread 0x0433fa thru 0x044501 (0x1108 bytes) 000056[1] 1774529079.633028 read 0x62c050 thru 0x62d7e6 (0x1797 bytes) 000058[1] 1774529079.733871 read 0x008c83 thru 0x00a767 (0x1ae5 bytes) 000060[0] 1774529079.834114 mapwrite 0x749cbf thru 0x74aa8c (0xdce bytes) 000061[0] 1774529079.939883 mapread 0x5882d4 thru 0x588960 (0x68d bytes) 000062[0] 1774529079.948208 mapwrite 0x0c9410 thru 0x0c9851 (0x442 bytes) 000065[1] 1774529080.130055 read 0x84aabe thru 0x84adf1 (0x334 bytes) 000067[0] 1774529080.221378 read 0x799c6c thru 0x79a7eb (0xb80 bytes) 000068[1] 1774529080.262192 write 0x79e831 thru 0x7a01fc (0x19cc bytes) 000070[0] 1774529080.312457 mapwrite 0x0d5ef8 thru 0x0d61e8 (0x2f1 bytes) 000072[1] 1774529080.491170 trunc from 0x85c27f to 0x528ca9 (0x3335d5 bytes) 000073[1] 1774529080.552054 mapread 0x0a6b8d thru 0x0a77ff (0xc73 bytes) 000074[0] 1774529080.568459 read 0x0eb1da thru 0x0ec549 (0x1370 bytes) 000078[1] 1774529080.697026 trunc from 0x528caa to 0x027178 (0x501b31 bytes) 000080[1] 1774529080.781382 write 0x9a3536 thru 0x9a3aad (0x578 bytes) 000081[1] 1774529080.803051 mapwrite 0x4807ca thru 0x481f92 (0x17c9 bytes) 000082[1] 1774529081.001437 mapread 0x74201b thru 0x743f27 (0x1f0d bytes) 000083[1] 1774529081.021343 write 0x0979e9 thru 0x09892f (0xf47 bytes) 000084[0] 1774529081.042040 read 0x9a1d50 thru 0x9a21c6 (0x477 bytes) 000085[1] 1774529081.071137 read 0x199ec7 thru 0x19a309 (0x443 bytes) truncating to largest ever: 0x841463 000087[1] 1774529081.234355 trunc from 0x9a3aae to 0x841462 (0x16264b bytes) 000088[0] 1774529081.297329 mapread 0x67bd09 thru 0x67bf2d (0x225 bytes) 000089[0] 1774529081.393676 mapread 0x1af299 thru 0x1afc09 (0x971 bytes) 000091[1] 1774529081.429876 mapread 0x7af620 thru 0x7b14f3 (0x1ed4 bytes) 000092[1] 1774529081.451614 write 0x48e9a8 thru 0x490551 (0x1baa bytes) 000095[1] 1774529081.538400 trunc from 0x841463 to 0x5c1fd3 (0x27f48f bytes) 000096[0] 1774529081.591397 mapread 0x0b4429 thru 0x0b53f4 (0xfcc bytes) 000097[1] 1774529081.641362 read 0x32b2d2 thru 0x32b890 (0x5bf bytes) 000098[1] 1774529081.726116 trunc from 0x5c1fd4 to 0x65a590 (0x985bd bytes) 000099[1] 1774529081.773980 write 0x4755ec thru 0x475c42 (0x657 bytes) 000103[0] 1774529081.895822 mapread 0x44ce35 thru 0x44d7c0 (0x98c bytes) 000106[1] 1774529081.974132 read 0x5b0ce5 thru 0x5b1243 (0x55f bytes) 000107[1] 1774529081.974769 mapread 0x4c13a4 thru 0x4c14b0 (0x10d bytes) 000108[1] 1774529081.982468 mapread 0x19926e thru 0x1999c6 (0x759 bytes) 000109[1] 1774529081.999149 write 0x0ad8a7 thru 0x0af310 (0x1a6a bytes) 000110[0] 1774529082.099195 trunc from 0x65a591 to 0x3544ea (0x3060a6 bytes) 000111[1] 1774529082.171006 read 0x1a55ba thru 0x1a7526 (0x1f6d bytes) 000114[0] 1774529082.211068 write 0x60cac7 thru 0x60d66a (0xba4 bytes) 000115[1] 1774529082.231904 read 0x489ede thru 0x48ad0a (0xe2d bytes) 000117[1] 1774529082.287572 read 0x248b20 thru 0x249c43 (0x1124 bytes) 000118[0] 1774529082.325229 write 0x1829fb thru 0x18437e (0x1984 bytes) 000119[1] 1774529082.398976 mapwrite 0x85ddc0 thru 0x85f4b0 (0x16f1 bytes) 000121[1] 1774529082.620796 read 0x6855ad thru 0x686aba (0x150e bytes) 000122[1] 1774529082.649159 mapread 0x55ffa1 thru 0x560fa4 (0x1004 bytes) 000123[1] 1774529082.662979 read 0x6c762f thru 0x6c925f (0x1c31 bytes) 000124[0] 1774529082.663870 mapwrite 0x443fa6 thru 0x44471d (0x778 bytes) 000125[1] 1774529082.775695 mapread 0x0ba85f thru 0x0bbebb (0x165d bytes) 000127[0] 1774529082.853955 mapwrite 0x51acae thru 0x51be94 (0x11e7 bytes) 000128[0] 1774529082.990194 trunc from 0x85f4b1 to 0x0a2e2a (0x7bc686 bytes) 000130[0] 1774529083.159401 mapwrite 0x900356 thru 0x901d48 (0x19f3 bytes) 000131[1] 1774529083.389497 trunc from 0x901d49 to 0x82b69a (0xd66ae bytes) 000132[1] 1774529083.466292 mapwrite 0x615904 thru 0x615fc4 (0x6c1 bytes) 000133[0] 1774529083.534454 read 0x6fb0b0 thru 0x6fc3a1 (0x12f2 bytes) 000134[0] 1774529083.560970 read 0x1a25c0 thru 0x1a3645 (0x1086 bytes) truncating to largest ever: 0x96f038 000135[0] 1774529083.686646 trunc from 0x82b69b to 0x96f037 (0x14399d bytes) 000136[0] 1774529083.761903 trunc from 0x96f038 to 0x31f076 (0x64ffc1 bytes) 000137[1] 1774529083.808101 read 0x11bbd6 thru 0x11c7bd (0xbe8 bytes) 000139[1] 1774529083.865100 read 0x29cb92 thru 0x29deff (0x136e bytes) 000141[0] 1774529083.908155 write 0x7cbdf6 thru 0x7cd86f (0x1a7a bytes) 000143[0] 1774529083.929785 mapread 0x79db36 thru 0x79e2a7 (0x772 bytes) 000144[0] 1774529083.958892 read 0x5a52ea thru 0x5a6e57 (0x1b6e bytes) 000145[0] 1774529083.985841 trunc from 0x7cd870 to 0x0cde9d (0x6ff9d2 bytes) 000149[0] 1774529084.113902 read 0x0a8e04 thru 0x0a972c (0x929 bytes) 000152[0] 1774529084.214183 write 0x20c60d thru 0x20c980 (0x374 bytes) 000153[1] 1774529084.217084 write 0x2bf66e thru 0x2c0ad7 (0x146a bytes) 000154[0] 1774529084.256355 write 0x7aee9c thru 0x7b06da (0x183f bytes) 000156[1] 1774529084.289652 mapread 0x058ab8 thru 0x059e28 (0x1371 bytes) 000159[1] 1774529084.351739 mapwrite 0x9a5653 thru 0x9a73a2 (0x1d50 bytes) 000164[1] 1774529084.538119 read 0x937da1 thru 0x9387ec (0xa4c bytes) 000165[1] 1774529084.547720 mapwrite 0x64d287 thru 0x64e279 (0xff3 bytes) 000167[0] 1774529084.647836 trunc from 0x9a73a3 to 0x60cf2c (0x39a476 bytes) 000168[1] 1774529084.727843 write 0x9215b4 thru 0x92181a (0x267 bytes) 000169[1] 1774529084.760619 mapwrite 0x2341a9 thru 0x234208 (0x60 bytes) 000170[0] 1774529084.861181 read 0x29ea1a thru 0x29fd5b (0x1342 bytes) 000172[0] 1774529084.907982 trunc from 0x92181b to 0x0a8cbe (0x878b5c bytes) 000174[0] 1774529084.944885 mapread 0x05968b thru 0x059e88 (0x7fe bytes) 000175[1] 1774529084.974168 mapread 0x027d0f thru 0x0290e1 (0x13d3 bytes) 000177[0] 1774529085.044688 read 0x0847ed thru 0x085e25 (0x1639 bytes) 000178[0] 1774529085.053658 trunc from 0x0a8cbf to 0x65d9fb (0x5b4d3d bytes) 000179[0] 1774529085.103011 trunc from 0x65d9fc to 0x78ea01 (0x131006 bytes) 000180[0] 1774529085.125652 mapwrite 0x1254b6 thru 0x125f36 (0xa81 bytes) 000183[1] 1774529085.268367 write 0x25590d thru 0x256d26 (0x141a bytes) 000186[1] 1774529085.343334 read 0x756046 thru 0x7563d0 (0x38b bytes) 000187[1] 1774529085.400843 read 0x188e9e thru 0x189524 (0x687 bytes) 000188[1] 1774529085.536309 write 0x6c508b thru 0x6c702e (0x1fa4 bytes) 000189[0] 1774529085.554858 trunc from 0x78ea02 to 0x6b4e71 (0xd9b90 bytes) 000195[1] 1774529085.697939 mapread 0x096da0 thru 0x098455 (0x16b6 bytes) 000199[0] 1774529085.771262 mapread 0x3e620f thru 0x3e81cc (0x1fbe bytes) 000200[1] 1774529085.798453 mapread 0x01e5f1 thru 0x01f48b (0xe9b bytes) 000202[0] 1774529085.827176 write 0x594e99 thru 0x595a99 (0xc01 bytes) 000203[1] 1774529085.851080 read 0x682892 thru 0x683a9c (0x120b bytes) 000204[0] 1774529085.875822 write 0x526d1e thru 0x528be7 (0x1eca bytes) 000207[0] 1774529085.917744 trunc from 0x6b4e72 to 0x693aeb (0x21386 bytes) 000208[0] 1774529085.963795 trunc from 0x693aec to 0x25788f (0x43c25c bytes) 000209[0] 1774529085.984359 write 0x7e1817 thru 0x7e3698 (0x1e82 bytes) 000215[0] 1774529086.044953 mapread 0x6f8463 thru 0x6f9377 (0xf15 bytes) 000216[1] 1774529086.082164 write 0x95d5c4 thru 0x95e291 (0xcce bytes) 000217[1] 1774529086.125987 mapread 0x165aa7 thru 0x16693e (0xe98 bytes) 000218[0] 1774529086.158495 read 0x43aaf6 thru 0x43ad73 (0x27e bytes) 000220[1] 1774529086.206890 read 0x276157 thru 0x276bb5 (0xa5f bytes) 000222[1] 1774529086.309111 mapread 0x179b69 thru 0x17a117 (0x5af bytes) 000227[0] 1774529086.359022 trunc from 0x95e292 to 0x7b5e0d (0x1a8484 bytes) 000229[0] 1774529086.481638 read 0x2400c7 thru 0x2416d2 (0x160c bytes) 000233[1] 1774529086.607255 mapread 0x652c1b thru 0x653e5f (0x1245 bytes) 000234[1] 1774529086.663107 write 0x9d3a1d thru 0x9d4f13 (0x14f7 bytes) 000236[0] 1774529086.697800 read 0x314398 thru 0x314714 (0x37d bytes) 000237[0] 1774529086.807200 mapread 0x584a24 thru 0x58532f (0x90c bytes) 000239[1] 1774529086.827684 read 0x3f263e thru 0x3f4454 (0x1e17 bytes) 000240[0] 1774529086.904701 mapread 0x921917 thru 0x9220d3 (0x7bd bytes) 000241[1] 1774529086.905272 mapwrite 0x96886b thru 0x969439 (0xbcf bytes) 000242[0] 1774529086.981738 mapwrite 0x4a10b9 thru 0x4a1146 (0x8e bytes) 000244[1] 1774529087.120008 read 0x12f743 thru 0x1301d5 (0xa93 bytes) 000245[0] 1774529087.193754 mapread 0x812cf3 thru 0x81467b (0x1989 bytes) 000247[0] 1774529087.306042 trunc from 0x9d4f14 to 0x535e05 (0x49f10e bytes) 000248[0] 1774529087.403234 trunc from 0x535e06 to 0x7c149e (0x28b699 bytes) 000250[0] 1774529087.442053 mapread 0x38f785 thru 0x390ac1 (0x133d bytes) 000252[1] 1774529087.462377 mapwrite 0x72f81a thru 0x7302b9 (0xaa0 bytes) 000254[1] 1774529087.555847 trunc from 0x7c149f to 0x8a40ae (0xe2c10 bytes) 000255[0] 1774529087.608020 mapread 0x54a4f0 thru 0x54c426 (0x1f37 bytes) 000256[1] 1774529087.646792 mapread 0x30b982 thru 0x30cc90 (0x130f bytes) 000258[0] 1774529087.798954 mapwrite 0x006a62 thru 0x008462 (0x1a01 bytes) 000260[1] 1774529087.909959 read 0x1bf365 thru 0x1bf6e0 (0x37c bytes) 000265[0] 1774529087.965138 read 0x27ccc2 thru 0x27e6b5 (0x19f4 bytes) 000266[0] 1774529088.064269 read 0x7b90d6 thru 0x7b9d65 (0xc90 bytes) 000267[1] 1774529088.076848 mapread 0x0a033f thru 0x0a0569 (0x22b bytes) 000268[0] 1774529088.177927 trunc from 0x8a40af to 0x57b82f (0x32887f bytes) 000269[1] 1774529088.290231 mapwrite 0x0e97d9 thru 0x0eb70f (0x1f37 bytes) 000270[1] 1774529088.488094 mapread 0x21aad2 thru 0x21ba4c (0xf7b bytes) 000272[0] 1774529088.621504 read 0x088b9e thru 0x08a5f4 (0x1a57 bytes) 000273[1] 1774529088.703754 read 0x33c214 thru 0x33c42c (0x219 bytes) 000275[0] 1774529088.773733 write 0x8925e6 thru 0x893759 (0x1174 bytes) 000276[1] 1774529088.836614 mapwrite 0x2009ca thru 0x20266f (0x1ca6 bytes) 000277[0] 1774529089.067051 write 0x97d95c thru 0x97e2a8 (0x94d bytes) 000278[0] 1774529089.099634 write 0x92e397 thru 0x92e7bb (0x425 bytes) 000279[0] 1774529089.135814 mapwrite 0x3b978e thru 0x3bb73f (0x1fb2 bytes) 000281[0] 1774529089.304891 mapwrite 0x802285 thru 0x80390f (0x168b bytes) 000282[0] 1774529089.425414 read 0x102344 thru 0x10381e (0x14db bytes) 000284[0] 1774529089.493234 read 0x96f2b9 thru 0x96f597 (0x2df bytes) 000285[0] 1774529089.506520 write 0x326646 thru 0x3274cc (0xe87 bytes) 000286[0] 1774529089.507377 write 0x5f63d0 thru 0x5f685d (0x48e bytes) 000287[0] 1774529089.518241 write 0x840ae1 thru 0x842936 (0x1e56 bytes) 000288[1] 1774529089.518963 mapwrite 0x00dff0 thru 0x00ea49 (0xa5a bytes) 000291[0] 1774529089.707227 trunc from 0x97e2a9 to 0x601038 (0x37d270 bytes) 000292[0] 1774529089.740627 read 0x285173 thru 0x2852c5 (0x153 bytes) 000293[1] 1774529089.772126 read 0x22ade3 thru 0x22c3bd (0x15db bytes) 000294[1] 1774529089.803944 mapwrite 0x027cab thru 0x0299bb (0x1d11 bytes) 000295[1] 1774529089.894637 read 0x43e5bd thru 0x43f065 (0xaa9 bytes) 000296[0] 1774529089.926059 write 0x884c15 thru 0x8864c1 (0x18ad bytes) 000297[0] 1774529089.938422 trunc from 0x8864c2 to 0x4fbcd8 (0x38a7e9 bytes) 000298[1] 1774529089.977085 write 0x5b3a51 thru 0x5b48ed (0xe9d bytes) 000300[1] 1774529090.037348 write 0x213aa3 thru 0x214fa2 (0x1500 bytes) 000302[0] 1774529090.085337 mapread 0x35e467 thru 0x35f5bf (0x1159 bytes) 000303[1] 1774529090.113101 trunc from 0x5b48ee to 0x7ee52f (0x239c42 bytes) 000307[1] 1774529090.176225 write 0x8681df thru 0x86923a (0x105c bytes) 000309[1] 1774529090.199320 mapread 0x600d6d thru 0x6026f4 (0x1988 bytes) 000311[0] 1774529090.279758 write 0x2fd226 thru 0x2fd36d (0x148 bytes) 000312[0] 1774529090.359175 mapwrite 0x1ff72e thru 0x1fff02 (0x7d5 bytes) 000313[1] 1774529090.459875 write 0x69bd16 thru 0x69db9a (0x1e85 bytes) 000315[1] 1774529090.513142 write 0x44099a thru 0x440c36 (0x29d bytes) 000319[1] 1774529090.568850 mapwrite 0x0bede4 thru 0x0c0c42 (0x1e5f bytes) 000320[1] 1774529090.654137 write 0x59f899 thru 0x5a15ce (0x1d36 bytes) 000323[0] 1774529090.690156 trunc from 0x86923b to 0x219851 (0x64f9e9 bytes) 000324[0] 1774529090.760890 mapread 0x15f7f4 thru 0x1613ec (0x1bf9 bytes) 000326[0] 1774529090.781132 write 0x57d225 thru 0x57d8b5 (0x691 bytes) 000328[0] 1774529090.782843 write 0x2798e4 thru 0x27ae62 (0x157f bytes) 000330[0] 1774529090.818507 write 0x2d1b72 thru 0x2d2d9d (0x122c bytes) 000332[0] 1774529090.867623 read 0x02768d thru 0x028c99 (0x160d bytes) 000333[0] 1774529090.966459 mapwrite 0x907f9f thru 0x908c84 (0xce6 bytes) 000334[1] 1774529091.120175 mapwrite 0x075014 thru 0x07661b (0x1608 bytes) 000337[1] 1774529091.201531 mapwrite 0x69287b thru 0x6947fc (0x1f82 bytes) truncating to largest ever: 0x975be2 000340[0] 1774529091.264120 trunc from 0x908c85 to 0x975be1 (0x6cf5d bytes) 000341[0] 1774529091.293068 read 0x79b4c7 thru 0x79ca53 (0x158d bytes) 000342[0] 1774529091.295582 read 0x2ea8eb thru 0x2ec190 (0x18a6 bytes) 000344[1] 1774529091.353090 read 0x406851 thru 0x407fe7 (0x1797 bytes) 000347[0] 1774529091.376829 trunc from 0x975be2 to 0x36f2fc (0x6068e5 bytes) 000348[1] 1774529091.400732 read 0x1fdab7 thru 0x1ff45e (0x19a8 bytes) 000350[1] 1774529091.428439 trunc from 0x36f2fd to 0x1fdf95 (0x171367 bytes) 000352[1] 1774529091.493137 trunc from 0x1fdf96 to 0x946e1d (0x748e88 bytes) 000353[1] 1774529091.536184 mapread 0x2210f3 thru 0x222017 (0xf25 bytes) 000355[0] 1774529091.643516 read 0x73acac thru 0x73bab5 (0xe0a bytes) 000357[1] 1774529091.701984 read 0x164b2d thru 0x16663c (0x1b10 bytes) 000358[1] 1774529091.789138 mapread 0x042284 thru 0x0435d3 (0x1350 bytes) 000360[1] 1774529091.848455 mapread 0x0b828d thru 0x0b9982 (0x16f6 bytes) 000361[0] 1774529091.867023 read 0x1117f6 thru 0x11331e (0x1b29 bytes) 000362[1] 1774529091.900225 write 0x7349c9 thru 0x7360dd (0x1715 bytes) 000364[1] 1774529091.943108 mapwrite 0x2c5395 thru 0x2c6a6e (0x16da bytes) 000365[1] 1774529091.972505 mapwrite 0x258871 thru 0x258d15 (0x4a5 bytes) 000367[1] 1774529092.023264 mapwrite 0x01125d thru 0x012bf1 (0x1995 bytes) 000368[1] 1774529092.120709 mapread 0x69799d thru 0x698223 (0x887 bytes) 000370[0] 1774529092.156866 mapwrite 0x1b83cf thru 0x1b9bd1 (0x1803 bytes) 000372[1] 1774529092.307383 read 0x824de4 thru 0x826614 (0x1831 bytes) 000373[1] 1774529092.331102 mapwrite 0x8732d8 thru 0x873a4a (0x773 bytes) 000374[1] 1774529092.390388 write 0x236219 thru 0x237e89 (0x1c71 bytes) 000376[1] 1774529092.425422 trunc from 0x946e1e to 0x883105 (0xc3d18 bytes) 000377[0] 1774529092.457920 read 0x135fb3 thru 0x13606c (0xba bytes) 000378[1] 1774529092.526342 mapwrite 0x23bf2e thru 0x23c89d (0x970 bytes) 000379[0] 1774529092.600012 read 0x3c9d2c thru 0x3caa4e (0xd23 bytes) 000382[1] 1774529092.621348 read 0x044717 thru 0x044b2f (0x419 bytes) 000386[1] 1774529092.733998 trunc from 0x883106 to 0x2fcfb4 (0x586151 bytes) 000388[1] 1774529092.774025 trunc from 0x2fcfb5 to 0x8d1e30 (0x5d4e7c bytes) 000389[1] 1774529092.788111 trunc from 0x8d1e31 to 0x6cb081 (0x206daf bytes) 000390[1] 1774529092.812342 trunc from 0x6cb082 to 0x36ef9a (0x35c0e7 bytes) 000391[1] 1774529092.826565 write 0x284d42 thru 0x285baf (0xe6e bytes) 000393[0] 1774529092.848587 trunc from 0x36ef9b to 0x616ebf (0x2a7f25 bytes) 000395[0] 1774529092.903011 trunc from 0x616ec0 to 0x4e4fa9 (0x131f16 bytes) 000396[0] 1774529092.939229 trunc from 0x4e4faa to 0x22b973 (0x2b9636 bytes) 000398[1] 1774529092.951209 write 0x0077b5 thru 0x007e16 (0x662 bytes) 000399[1] 1774529092.971200 mapread 0x214675 thru 0x215610 (0xf9c bytes) 000400[1] 1774529093.002747 mapwrite 0x6013cc thru 0x6029da (0x160f bytes) 000402[0] 1774529093.116102 mapread 0x5653ed thru 0x5664a1 (0x10b5 bytes) 000404[1] 1774529093.152195 mapwrite 0x338a84 thru 0x339dc6 (0x1343 bytes) 000406[1] 1774529093.219069 mapread 0x5f1668 thru 0x5f2492 (0xe2b bytes) 000408[0] 1774529093.234180 read 0x462550 thru 0x4633a1 (0xe52 bytes) 000409[1] 1774529093.276205 mapwrite 0x1d831c thru 0x1d8b74 (0x859 bytes) 000410[1] 1774529093.324141 mapread 0x04b4be thru 0x04bdf1 (0x934 bytes) 000413[0] 1774529093.353982 trunc from 0x6029db to 0x665efa (0x63520 bytes) 000414[0] 1774529093.396869 trunc from 0x665efb to 0x197bf6 (0x4ce304 bytes) 000415[1] 1774529093.421664 write 0x034d8b thru 0x0354b8 (0x72e bytes) 000419[0] 1774529093.485792 mapread 0x0fae2d thru 0x0fbcc8 (0xe9c bytes) 000421[0] 1774529093.525149 trunc from 0x197bf7 to 0x7072e1 (0x56f6eb bytes) 000422[0] 1774529093.552324 trunc from 0x7072e2 to 0x2275ef (0x4dfcf2 bytes) 000423[1] 1774529093.583523 write 0x19b04e thru 0x19b4f1 (0x4a4 bytes) 000424[0] 1774529093.605512 mapread 0x046c43 thru 0x047bf7 (0xfb5 bytes) 000427[0] 1774529093.665795 trunc from 0x2275f0 to 0x638c0a (0x41161b bytes) 000428[1] 1774529093.697740 write 0x2fa7f9 thru 0x2fb7fa (0x1002 bytes) 000431[0] 1774529093.766190 mapwrite 0x62f107 thru 0x62fb02 (0x9fc bytes) 000432[1] 1774529093.909108 write 0x73a0d1 thru 0x73aea0 (0xdd0 bytes) 000434[1] 1774529093.962202 mapwrite 0x6e8c6b thru 0x6ea319 (0x16af bytes) 000435[1] 1774529094.033474 mapread 0x2a6595 thru 0x2a6f93 (0x9ff bytes) 000436[0] 1774529094.037765 read 0x0b2a9f thru 0x0b3ce4 (0x1246 bytes) 000437[1] 1774529094.065470 read 0x5a6b3b thru 0x5a877e (0x1c44 bytes) 000439[0] 1774529094.113418 mapwrite 0x9f78b4 thru 0x9f7b26 (0x273 bytes) 000441[0] 1774529094.465647 mapread 0x1e60fa thru 0x1e6f1c (0xe23 bytes) 000442[1] 1774529094.492013 read 0x8c5667 thru 0x8c61c0 (0xb5a bytes) 000444[1] 1774529094.590786 trunc from 0x9f7b27 to 0x45b78b (0x59c39b bytes) 000448[0] 1774529094.661046 mapwrite 0x27bb41 thru 0x27d7da (0x1c9a bytes) 000451[0] 1774529094.791755 mapread 0x21bdf6 thru 0x21c214 (0x41f bytes) 000453[0] 1774529094.848822 mapread 0x3ddd41 thru 0x3df106 (0x13c6 bytes) 000454[0] 1774529094.879117 write 0x78bec7 thru 0x78c57b (0x6b5 bytes) 000457[0] 1774529094.933240 trunc from 0x78c57c to 0x4201ee (0x36c38d bytes) 000458[1] 1774529094.959876 mapwrite 0x6a5eb8 thru 0x6a7e7e (0x1fc7 bytes) 000460[0] 1774529095.194322 read 0x115e34 thru 0x11628b (0x458 bytes) 000461[0] 1774529095.261113 write 0x0743c1 thru 0x0761db (0x1e1b bytes) 000462[0] 1774529095.419481 mapwrite 0x0562c4 thru 0x0562f2 (0x2f bytes) 000471[0] 1774529095.637782 mapread 0x0f459f thru 0x0f5d29 (0x178b bytes) 000475[1] 1774529095.712090 mapread 0x07c62c thru 0x07dff3 (0x19c8 bytes) 000476[1] 1774529095.748229 mapread 0x049ef9 thru 0x04a97c (0xa84 bytes) 000479[0] 1774529095.760660 mapread 0x5ac85b thru 0x5ad353 (0xaf9 bytes) 000480[1] 1774529095.761291 mapread 0x2dc914 thru 0x2dde26 (0x1513 bytes) 000481[0] 1774529095.784744 trunc from 0x6a7e7f to 0x86c8cd (0x1c4a4f bytes) 000482[0] 1774529095.880169 mapwrite 0x12c7c9 thru 0x12c8ee (0x126 bytes) 000484[1] 1774529095.978996 read 0x001607 thru 0x002e35 (0x182f bytes) 000485[0] 1774529096.021875 read 0x80452f thru 0x805099 (0xb6b bytes) 000488[0] 1774529096.073852 mapwrite 0x7b8a3a thru 0x7b9f0b (0x14d2 bytes) 000489[1] 1774529096.118267 write 0x9e22dd thru 0x9e293c (0x660 bytes) 000491[0] 1774529096.126651 read 0x98af2d thru 0x98c068 (0x113c bytes) 000492[0] 1774529096.173035 mapread 0x27af87 thru 0x27c414 (0x148e bytes) 000494[1] 1774529096.184420 mapwrite 0x753a95 thru 0x7554a4 (0x1a10 bytes) 000496[1] 1774529096.402933 mapread 0x4366ac thru 0x4372bd (0xc12 bytes) 000497[1] 1774529096.403715 trunc from 0x9e293d to 0x8f0795 (0xf21a7 bytes) 000498[0] 1774529096.447058 mapwrite 0x34f3b3 thru 0x34fb3c (0x78a bytes) All operations completed A-OK! PASS 16b (25s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 08:45:00 (1774529100) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 08:45:03 (1774529103) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.202075 s, 10.4 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.154047 s, 13.6 MB/s ldlm.namespaces.MGC192.168.202.160@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff97ae03c84000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff97ae0a7fe800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff97ae0a7fe800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff97ae0a7fe800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.29386 s, 14.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.02234 s, 51.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.82514 s, 57.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.72455 s, 22.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.56804 s, 18.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.81707 s, 37.2 MB/s PASS 16d (34s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 08:45:37 (1774529137) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.197346 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100734 s, 10.4 MB/s PASS 16e (5s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 08:45:42 (1774529142) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 08:46:07 (1774529167) TIMEOUT PASS 16g (26s) == sanityn test 16i: read after truncate file ============ 08:46:32 (1774529192) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (5s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 08:46:37 (1774529197) fail_loc=0x8000030a PASS 17 (7s) == sanityn test 18: mmap sanity check =========================================================================================== 08:46:44 (1774529204) mmap test1: basic mmap operation (PASS, 0.11547s) mmap test2: MAP_PRIVATE not write back (PASS, 0.045174s) mmap test3: concurrent mmap ops on two nodes (PASS, 10.509s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.2654s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.094446s) mmap test6: check mmap write/read content on two nodes (PASS, 0.084789s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0436s) PASS 18 (20s) == sanityn test 19: test concurrent uncached read races ========================================================================= 08:47:04 (1774529224) oleg260-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg260-client: oleg260-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 08:47:06 (1774529226) PASS 20 (4s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 08:47:10 (1774529230) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (3s) == sanityn test 23: others should see updated atime while another read============================================================== 08:47:14 (1774529234) now is 1774529235 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 new atime is 1774529296 PASS 23 (66s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 08:48:19 (1774529299) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 08:48:22 (1774529302) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 08:48:25 (1774529305) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (4s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 08:48:29 (1774529309) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 08:48:30 (1774529310) utime: good mknod times 1774529310 <= 1774529311 <= 1774529311 for /mnt/lustre/f26a utime: good mknod times 1774529310 <= 1774529311 <= 1774529311 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 08:48:35 (1774529315) PASS 26b (5s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 08:48:42 (1774529322) 0 PASS 30 (6s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 08:48:48 (1774529328) fail_loc=0x314 PASS 31a (5s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 08:48:53 (1774529333) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg260-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.160@tcp:/lustre 7542784 13312 7525376 1% /mnt/lustre2 PASS 31b (17s) == sanityn test 31r: open-rename(replace) race =========== 08:49:10 (1774529350) -rw-r--r-- 1 root root 0 Mar 26 08:49 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (7s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 08:49:18 (1774529358) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 08:49:20 (1774529360) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 08:49:22 (1774529362) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 08:49:24 (1774529364) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (2s) == sanityn test 34: no lock timeout under IO ============= 08:49:26 (1774529366) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg260-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97ae03c84000.ost_server_uuid,osc.lustre-OST0000-osc-ffff97ae0a7fe800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff97ae03c84000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff97ae0a7fe800.ost_server_uuid in IDLE state after 0 sec oleg260-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff97ae03c84000.ost_server_uuid,osc.lustre-OST0001-osc-ffff97ae0a7fe800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff97ae03c84000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff97ae0a7fe800.ost_server_uuid in FULL state after 0 sec oleg260-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97ae03c84000.ost_server_uuid,osc.lustre-OST0000-osc-ffff97ae0a7fe800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff97ae03c84000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff97ae0a7fe800.ost_server_uuid in IDLE state after 0 sec oleg260-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff97ae03c84000.ost_server_uuid,osc.lustre-OST0001-osc-ffff97ae0a7fe800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff97ae03c84000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff97ae0a7fe800.ost_server_uuid in FULL state after 0 sec oleg260-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97ae03c84000.ost_server_uuid,osc.lustre-OST0000-osc-ffff97ae0a7fe800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff97ae03c84000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff97ae0a7fe800.ost_server_uuid in IDLE state after 0 sec oleg260-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff97ae03c84000.ost_server_uuid,osc.lustre-OST0001-osc-ffff97ae0a7fe800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff97ae03c84000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff97ae0a7fe800.ost_server_uuid in FULL state after 0 sec PASS 34 (99s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 08:51:05 (1774529465) fail_loc=0x80000317 Race attempt 0 Wait for 44670 44753 for 60 sec... - open/close 1099 (time 1774529479.60 total 10.01 last 109.79) - open/close 2252 (time 1774529489.61 total 20.02 last 115.24) - open/close 3500 (time 1774529499.61 total 30.02 last 124.76) total: 4000 open/close in 33.89 seconds: 118.02 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.160@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre 192.168.202.160@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre2 PASS 35 (67s) == sanityn test 36: handle ESTALE/open-unlink correctly == 08:52:12 (1774529532) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg260-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.65782 s, 79.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7525376) after_dd(7465984) after(7517184) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.603409 s, 86.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7517184) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.530402 s, 98.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.458599 s, 114 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.477439 s, 110 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.48983 s, 107 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.441105 s, 119 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.478759 s, 110 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.471274 s, 111 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7525376) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.540113 s, 97.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7521280) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.452081 s, 116 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7523328) after_dd(7472128) after(7523328) PASS 36 (254s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 08:56:26 (1774529786) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 - create 2981 (time 1774529798.63 total 10.00 last 298.07) - create 5871 (time 1774529808.63 total 20.00 last 288.96) - create 9099 (time 1774529818.64 total 30.00 last 322.74) total: 10000 create in 33.03 seconds: 302.78 ops/second PASS 37 (38s) == sanityn test 39a: test from 11063 ============================================================================================ 08:57:05 (1774529825) repeat after cancel_lru_locks PASS 39a (4s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 08:57:08 (1774529828) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 08:57:11 (1774529831) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 08:57:15 (1774529835) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 40a: pdirops: create vs others ======================================================================== 08:57:18 (1774529838) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (10s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 08:57:28 (1774529848) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (10s) == sanityn test 40c: pdirops: link and others ======================================================================== 08:57:39 (1774529859) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (10s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 08:57:48 (1774529868) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (9s) == sanityn test 40e: pdirops: rename and others ======================================================================== 08:57:57 (1774529877) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (9s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 08:58:06 (1774529886) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (7s) == sanityn test 41b: pdirops: create vs create ======================================================================== 08:58:13 (1774529893) open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 08:58:19 (1774529899) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 08:58:25 (1774529905) Conflict PASS 41d (7s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 08:58:32 (1774529912) Conflict PASS 41e (7s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 08:58:39 (1774529919) Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 08:58:45 (1774529925) Conflict PASS 41g (7s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 08:58:52 (1774529932) Conflict PASS 41h (7s) == sanityn test 41i: reint_open: create vs create ======== 08:58:59 (1774529939) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (758s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 09:11:37 (1774530697) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (6s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 09:11:43 (1774530703) open: File exists Conflict PASS 42b (6s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 09:11:49 (1774530709) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (6s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 09:11:55 (1774530715) Conflict PASS 42d (6s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 09:12:01 (1774530721) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (6s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 09:12:07 (1774530727) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 09:12:12 (1774530732) Conflict PASS 42g (6s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 09:12:18 (1774530738) Conflict PASS 42h (6s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 09:12:24 (1774530744) PASS 43a (18s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 09:12:42 (1774530762) Conflict PASS 43b (6s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 09:12:48 (1774530768) Conflict PASS 43c (6s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 09:12:54 (1774530774) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 09:12:59 (1774530779) Conflict PASS 43e (6s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 09:13:05 (1774530785) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (6s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 09:13:11 (1774530791) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (6s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 09:13:17 (1774530797) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 09:13:22 (1774530802) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 09:13:24 (1774530804) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (50s) == sanityn test 43k: unlink vs create ==================== 09:14:14 (1774530854) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (480s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 09:22:14 (1774531334) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu Mar 26 09:22:18 EDT 2026 PASS 44a (6s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 09:22:20 (1774531340) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 09:22:25 (1774531345) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (6s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 09:22:31 (1774531351) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 09:22:36 (1774531356) Conflict PASS 44e (6s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 09:22:42 (1774531362) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 09:22:47 (1774531367) Conflict PASS 44g (6s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 09:22:53 (1774531373) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 09:22:58 (1774531378) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 09:22:59 (1774531379) PASS 45a (27s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 09:23:26 (1774531406) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 09:23:31 (1774531411) Conflict PASS 45c (6s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 09:23:37 (1774531417) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 09:23:42 (1774531422) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 09:23:47 (1774531427) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (6s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 09:23:53 (1774531433) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 09:23:58 (1774531438) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 09:24:03 (1774531443) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 09:24:04 (1774531444) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (440s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 09:31:24 (1774531884) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 09:31:29 (1774531889) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 09:31:34 (1774531894) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 09:31:39 (1774531899) Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 09:31:44 (1774531904) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 09:31:49 (1774531909) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 09:31:54 (1774531914) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 09:31:59 (1774531919) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 09:32:04 (1774531924) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 09:32:05 (1774531925) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 09:32:06 (1774531926) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 09:32:07 (1774531927) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 09:32:08 (1774531928) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 09:32:09 (1774531929) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 09:32:10 (1774531930) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 09:32:11 (1774531931) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 09:32:12 (1774531932) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00502378 s, 2.0 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 09:32:19 (1774531939) 0+1 records in 0+1 records out 158 bytes copied, 0.00410757 s, 38.5 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242459) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 09:32:23 (1774531943) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00267944 s, 382 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127013 s, 82.6 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 09:32:38 (1774531958) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00703464 s, 146 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 09:32:44 (1774531964) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138814 s, 75.5 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 09:32:47 (1774531967) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 09:32:51 (1774531971) llite.lustre-ffff97ae03c84000.xattr_cache=1 llite.lustre-ffff97ae0a7fe800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:32:52.000000000 -0400 Modify: 2026-03-26 09:32:52.000000000 -0400 Change: 2026-03-26 09:32:52.000000000 -0400 Birth: 2026-03-26 09:32:51.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:32:57.000000000 -0400 Modify: 2026-03-26 09:32:57.000000000 -0400 Change: 2026-03-26 09:32:57.000000000 -0400 Birth: 2026-03-26 09:32:57.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:33:02.000000000 -0400 Modify: 2026-03-26 09:33:02.000000000 -0400 Change: 2026-03-26 09:33:02.000000000 -0400 Birth: 2026-03-26 09:33:02.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:33:02.000000000 -0400 Modify: 2026-03-26 09:33:03.000000000 -0400 Change: 2026-03-26 09:33:03.000000000 -0400 Birth: 2026-03-26 09:33:02.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:33:08.000000000 -0400 Modify: 2026-03-26 09:33:08.000000000 -0400 Change: 2026-03-26 09:33:08.000000000 -0400 Birth: 2026-03-26 09:33:08.000000000 -0400 File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:33:08.000000000 -0400 Modify: 2026-03-26 09:33:08.000000000 -0400 Change: 2026-03-26 09:33:08.000000000 -0400 Birth: 2026-03-26 09:33:08.000000000 -0400 PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 09:33:15 (1774531995) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 09:33:22 (1774532002) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 09:33:30 (1774532010) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 09:33:42 (1774532022) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 60: Verify data_version behaviour ======== 09:33:52 (1774532032) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 09:33:54 (1774532034) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 09:33:56 (1774532036) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 09:33:58 (1774532038) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00269422 s, 24.3 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 09:33:59 (1774532039) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0029227 s, 22.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 09:34:00 (1774532040) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 09:34:01 (1774532041) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00336415 s, 19.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 09:34:02 (1774532042) llite.lustre-ffff97ae03c84000.xattr_cache=1 llite.lustre-ffff97ae0a7fe800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 09:34:04 (1774532044) llite.lustre-ffff97ae03c84000.xattr_cache=1 llite.lustre-ffff97ae0a7fe800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 09:34:06 (1774532046) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00212184 s, 483 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0021581 s, 474 kB/s 252505: get lock2 252508: get lock1 252505: done 252505: get lock1 252508: done 252508: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252505: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 252508: cannot get lock2: Resource deadlock avoided 252508: exit rc=0 252505: done 252505: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 09:34:12 (1774532052) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0151909 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844509 s, 124 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253081 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 09:34:19 (1774532059) opening files: ........ closing files: ........ PASS 76 (16s) == sanityn test 77a: check FIFO NRS policy =============== 09:34:35 (1774532075) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.0886167 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167254 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465386 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0543064 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0691468 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554244 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710237 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805817 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844972 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715294 s, 14.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713627 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078701 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712611 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938875 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997244 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924724 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817603 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135449 s, 77.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182679 s, 57.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013059 s, 80.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198051 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193999 s, 54.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142253 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957542 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150872 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192304 s, 54.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637435 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181712 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00344889 s, 304 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00438212 s, 239 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147478 s, 71.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807787 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00533425 s, 197 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 09:34:37 (1774532077) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.113357 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260741 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442274 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638361 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620307 s, 16.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707203 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727278 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828426 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664259 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706816 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573329 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552057 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855157 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819232 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738004 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074863 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685039 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119103 s, 88.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160825 s, 65.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135129 s, 77.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130027 s, 80.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114056 s, 91.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146928 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141127 s, 74.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813082 s, 129 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074035 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618542 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106596 s, 98.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937029 s, 112 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00444005 s, 236 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880077 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0027972 s, 375 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00357569 s, 293 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.112619 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330647 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239893 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434141 s, 24.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263744 s, 39.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0494367 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531518 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597311 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051833 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374091 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352603 s, 29.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708628 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911036 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968546 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0896291 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106301 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973994 s, 10.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131745 s, 79.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017219 s, 60.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116588 s, 89.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966741 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133019 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00299914 s, 350 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018453 s, 56.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181798 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00510346 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00387703 s, 270 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115506 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00373262 s, 281 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286336 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676252 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277046 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129739 s, 80.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 09:34:41 (1774532081) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.105332 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319987 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031799 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337513 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681509 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668931 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772308 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864051 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960655 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977921 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0859663 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790777 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101207 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101848 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869543 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871294 s, 12.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100275 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951267 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137184 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173655 s, 60.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136444 s, 76.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00510599 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0028246 s, 371 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136195 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00363386 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903753 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201519 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243696 s, 43.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776965 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269427 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117399 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207428 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00460913 s, 227 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1196 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293633 s, 35.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379961 s, 27.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344172 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322718 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522248 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617846 s, 17.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541119 s, 19.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711678 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892524 s, 11.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535694 s, 19.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917621 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821077 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643891 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980624 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995156 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949161 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153602 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167595 s, 62.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168373 s, 62.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014058 s, 74.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220424 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260374 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100001 s, 105 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200452 s, 52.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113471 s, 92.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199701 s, 52.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921066 s, 114 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068785 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134391 s, 78.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297731 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00495992 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135289 s, 77.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.141035 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339713 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314542 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403034 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0495301 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391096 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433209 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655089 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060221 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04021 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736618 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0674602 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770379 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836343 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967695 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788136 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666139 s, 15.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=5 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156106 s, 67.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201986 s, 51.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185136 s, 56.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196702 s, 53.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171208 s, 61.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394995 s, 265 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155785 s, 67.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183996 s, 57.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116153 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00324043 s, 324 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215822 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277202 s, 37.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284459 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00481018 s, 218 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272354 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00465593 s, 225 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (5s) == sanityn test 77d: check TRR nrs policy ================ 09:34:46 (1774532086) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.122106 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199993 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021848 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049502 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594511 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478987 s, 21.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660871 s, 15.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557039 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559368 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640882 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644503 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954139 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0962993 s, 10.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945434 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987159 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763562 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931486 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130369 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189834 s, 55.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282811 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221555 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274606 s, 38.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269519 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188912 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215981 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217881 s, 48.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292281 s, 35.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0390597 s, 26.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365236 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365167 s, 28.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285285 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225592 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237273 s, 44.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.117682 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284691 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568733 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608049 s, 17.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838681 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775748 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669407 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0672882 s, 15.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747195 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865543 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0859624 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928334 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867084 s, 12.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803734 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880746 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836008 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967106 s, 10.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168672 s, 62.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140948 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157079 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128323 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135555 s, 77.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145333 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142555 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00327395 s, 320 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167087 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394275 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00390344 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230566 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710553 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140496 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00491412 s, 213 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00411592 s, 255 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.107397 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353987 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429508 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389214 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04882 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056084 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689533 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820138 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609442 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777118 s, 13.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753929 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077224 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094008 s, 11.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845029 s, 12.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883722 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102752 s, 10.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102592 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012667 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156976 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145913 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920245 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165037 s, 63.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140314 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158059 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996599 s, 105 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192136 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00508411 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00418886 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00362476 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00434074 s, 242 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00486442 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00332029 s, 316 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00320874 s, 327 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 09:34:50 (1774532090) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.60@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.0955272 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292988 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0498388 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673941 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627962 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562639 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772232 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624911 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073737 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683197 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0653683 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604461 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772641 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853349 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699051 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761944 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612163 s, 17.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139855 s, 75.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154572 s, 67.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129211 s, 81.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145589 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158505 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210864 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00371319 s, 282 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025328 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253481 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186904 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00382837 s, 274 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00522021 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123865 s, 84.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683745 s, 153 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974788 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00528731 s, 198 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.147678 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306018 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370053 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366898 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.035447 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584706 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529876 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577414 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790982 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0695924 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795417 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606984 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068455 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862911 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816235 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083141 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071713 s, 14.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130938 s, 80.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129052 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016756 s, 62.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137507 s, 76.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158391 s, 66.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193212 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190293 s, 55.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196095 s, 53.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217299 s, 48.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00408642 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136899 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00499092 s, 210 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936748 s, 112 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125405 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959153 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226496 s, 46.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.129075 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291246 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387324 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407594 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490901 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379333 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510816 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639891 s, 16.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832652 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831186 s, 12.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846318 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921235 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103869 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793267 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905034 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841656 s, 12.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912155 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141888 s, 73.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156492 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990451 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017472 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768029 s, 137 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239252 s, 43.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222499 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00381488 s, 275 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182626 s, 57.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203721 s, 51.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222732 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394897 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197343 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00248196 s, 422 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00347299 s, 302 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584509 s, 179 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.126126 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406555 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436038 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698205 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926777 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977994 s, 10.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111757 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115477 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10662 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108491 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117618 s, 8.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144932 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135349 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13367 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157134 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15029 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137898 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138243 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175986 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181817 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015063 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206933 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072964 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331522 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225749 s, 46.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225291 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906039 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00368124 s, 285 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733465 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220547 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144349 s, 72.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02172 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202463 s, 51.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (8s) == sanityn test 77f: check TBF JobID nrs policy ========== 09:34:58 (1774532098) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.163926 s, 102 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286264 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400232 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171558 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401888 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449383 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0443783 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398903 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509078 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514986 s, 20.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371188 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855601 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536035 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711741 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934174 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756656 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753055 s, 13.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952467 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183833 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116066 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124392 s, 84.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148644 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172867 s, 60.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179818 s, 58.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279924 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159265 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200217 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00331773 s, 316 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020412 s, 51.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232614 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226618 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00408255 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00487554 s, 215 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.125473 s, 134 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391521 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677056 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639784 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615913 s, 17.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060478 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0810231 s, 12.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0858822 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823505 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796046 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846236 s, 12.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921146 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870481 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842846 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808582 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079473 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082206 s, 12.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988533 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100385 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133534 s, 78.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181937 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107594 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137515 s, 76.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109732 s, 95.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162296 s, 64.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170995 s, 61.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177375 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00360705 s, 291 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077611 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222463 s, 47.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00454637 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00390866 s, 268 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00250912 s, 418 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.137237 s, 122 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238191 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587933 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607393 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648228 s, 16.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707658 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485563 s, 21.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654036 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732561 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650441 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867495 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816768 s, 12.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772658 s, 13.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723426 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0731377 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882181 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801548 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175194 s, 59.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240636 s, 43.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177616 s, 59.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192777 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00361394 s, 290 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221088 s, 47.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197101 s, 53.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223816 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186893 s, 56.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198571 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173445 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210593 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229673 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00302495 s, 347 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246241 s, 42.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00418154 s, 251 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.117609 s, 143 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372928 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493429 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496887 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478957 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413418 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0512432 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399116 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534611 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745404 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690627 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687378 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651002 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740477 s, 14.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare]1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078535 s, 13.4 MB/s [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0519839 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103157 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110255 s, 95.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158495 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127916 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019682 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210938 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156162 s, 67.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142606 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169679 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0043323 s, 242 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01968 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686526 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180722 s, 58.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00337229 s, 311 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128856 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651149 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00208517 s, 503 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (7s) == sanityn test 77g: Change TBF type directly ============ 09:35:05 (1774532105) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 09:35:08 (1774532108) oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg260-client: oleg260-server: ssh exited with exit code 2 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg260-client: oleg260-server: ssh exited with exit code 12 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg260-client: oleg260-server: ssh exited with exit code 12 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg260-client: oleg260-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 09:35:12 (1774532112) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (7s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 09:35:19 (1774532119) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.103639 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274247 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227344 s, 46.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227368 s, 46.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471386 s, 22.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353194 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420965 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04354 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575282 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0498771 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062594 s, 16.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044157 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454567 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913092 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773438 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877314 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856769 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163264 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150014 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170716 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151608 s, 69.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166547 s, 63.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00429193 s, 244 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163554 s, 64.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206732 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186444 s, 56.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135051 s, 77.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0034655 s, 303 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198417 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0051351 s, 204 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241342 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774416 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00368646 s, 284 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9138 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4343 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (47s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 09:36:06 (1774532166) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.649441 s, 25.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353326 s, 29.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490625 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532328 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341866 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424583 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564316 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361794 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440745 s, 23.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.350653 s, 3.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35605 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381569 s, 2.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.389167 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373616 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381416 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.406883 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.390952 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142777 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139983 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139866 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142397 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140371 s, 7.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149231 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142809 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146719 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14794 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146685 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148891 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15105 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150824 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150412 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152061 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152809 s, 6.9 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8294 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9527 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.650586 s, 25.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031769 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308977 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365402 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346288 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355384 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0486507 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407017 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381691 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35497 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369999 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382963 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365239 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40089 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395639 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.390972 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382447 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143877 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142564 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147601 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14925 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148059 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146962 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148129 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149835 s, 7.0 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155535 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148149 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15705 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16246 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150364 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158344 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158411 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156733 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8301 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9509 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (136s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 09:38:22 (1774532302) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.196702 s, 85.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244975 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485425 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568649 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225698 s, 46.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476349 s, 22.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402105 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034994 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775265 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759493 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988484 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860835 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106467 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1023 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103945 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103716 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100184 s, 10.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200184 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172595 s, 60.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186322 s, 56.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012199 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189943 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205623 s, 51.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223132 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220879 s, 47.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105934 s, 99.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200854 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101674 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230272 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570342 s, 184 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264948 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00383365 s, 274 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00383195 s, 274 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.975 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72969 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.60@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.60@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.203597 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253373 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432294 s, 24.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499303 s, 21.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051083 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572201 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431853 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470524 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375422 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335939 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559811 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0598089 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905609 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12302 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158287 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247781 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245978 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451378 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163735 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216859 s, 48.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210633 s, 49.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100889 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239982 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205666 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211461 s, 49.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822852 s, 127 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833276 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230697 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187605 s, 55.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904993 s, 116 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263639 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685658 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126224 s, 83.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93179 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72825 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.60@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.201493 s, 83.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262014 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581243 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568359 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456423 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496243 s, 21.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768861 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677579 s, 15.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462945 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734813 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061368 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906475 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087942 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095491 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982072 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118167 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975733 s, 10.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01598 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151769 s, 69.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127041 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121155 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130463 s, 80.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137824 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184434 s, 56.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014384 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017269 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190628 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700352 s, 150 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854096 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00384931 s, 272 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00436556 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00450701 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00176057 s, 596 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92248 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.87766 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.350776 s, 47.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348688 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675073 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557796 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809797 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662509 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0746125 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571907 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435776 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15117 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171332 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178208 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192253 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340205 s, 3.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31982 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.45063 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.479318 s, 2.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191599 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187349 s, 56.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162672 s, 64.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184828 s, 56.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191458 s, 54.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146361 s, 71.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235522 s, 44.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234269 s, 44.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156204 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021222 s, 49.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922215 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260244 s, 40.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859338 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00397844 s, 264 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220946 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278342 s, 37.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78524 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7311 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90537 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72986 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.647956 s, 25.9 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262779 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396986 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335561 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0641912 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710805 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546598 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060194 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661995 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352149 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349476 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359551 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369627 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369732 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.390267 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382344 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399314 s, 2.6 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128571 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124982 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011698 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116597 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128705 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151195 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170275 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210674 s, 49.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186947 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00369566 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294558 s, 35.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020724 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210925 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226589 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337255 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00401689 s, 261 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6543 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4289 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.194539 s, 86.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335628 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455758 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447522 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409261 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308234 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499552 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344332 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603646 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321406 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675659 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0754179 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975423 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101161 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102902 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944367 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758249 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138858 s, 75.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145021 s, 72.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015366 s, 68.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161033 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178841 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182051 s, 57.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185054 s, 56.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189261 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274428 s, 38.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193145 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393344 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236362 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024757 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246705 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256338 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276831 s, 37.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91191 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73305 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.353068 s, 47.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033679 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222012 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200132 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335199 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441524 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332429 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339294 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345504 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1567 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161585 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172233 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184063 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339838 s, 3.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346737 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.362003 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357708 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103073 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300104 s, 34.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01778 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169345 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202571 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243375 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211734 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272273 s, 38.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226695 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252779 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272074 s, 38.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269862 s, 38.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265355 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279884 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281941 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304226 s, 34.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77508 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73279 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76327 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73063 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (351s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 09:44:13 (1774532653) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 09:44:16 (1774532656) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg260-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.67851 s, 285 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg260-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0065 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 09:45:07 (1774532707) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.144348 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274245 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360803 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043113 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698292 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697128 s, 15.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906341 s, 11.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703958 s, 14.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076383 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790544 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995933 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864308 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880403 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807211 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781516 s, 13.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830619 s, 12.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ae03c84000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103173 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170675 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173122 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017635 s, 59.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169369 s, 61.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261529 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173351 s, 60.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155371 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162206 s, 64.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216789 s, 48.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207685 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222742 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00333052 s, 315 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247436 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252258 s, 41.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ae03c84000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299426 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00156562 s, 670 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg260-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg260-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97495 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96833 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.159512 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449354 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475623 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440922 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942372 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650934 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848928 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644539 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773122 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621321 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865388 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133563 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122491 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236758 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222127 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316659 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.327231 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172016 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140023 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00398393 s, 263 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176237 s, 59.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204384 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980248 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228028 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215142 s, 48.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205326 s, 51.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018064 s, 58.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204666 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451444 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226716 s, 46.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749393 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0046635 s, 225 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00196745 s, 533 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg260-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg260-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92256 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96901 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (66s) == sanityn test 77o: Changing rank should not panic ====== 09:46:13 (1774532773) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 09:46:17 (1774532777) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (39s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 09:46:56 (1774532816) mds.MDS.mdt.nrs_policies=tbf oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg260-client: oleg260-server: ssh exited with exit code 36 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg260-client: oleg260-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (11s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 09:47:07 (1774532827) oleg260-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg260-client: oleg260-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 09:47:11 (1774532831) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:47:11.000000000 -0400 Modify: 2026-03-26 09:47:11.000000000 -0400 Change: 2026-03-26 09:47:11.000000000 -0400 Birth: 2026-03-26 09:47:11.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 09:47:23 (1774532843) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 09:47:24 (1774532844) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 09:47:25 (1774532845) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 09:47:26 (1774532846) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 09:47:27 (1774532847) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 09:47:28 (1774532848) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7141 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7141 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 09:47:31 (1774532851) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 09:47:32 (1774532852) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 09:47:39 (1774532859) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 09:47:40 (1774532860) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 09:47:41 (1774532861) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 09:47:42 (1774532862) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2646 0xa56 0 1 2627 0xa43 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2647 0xa57 0 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 09:47:51 (1774532871) osc.lustre-OST0000-osc-ffff97ae03c84000.idle_timeout=0 osc.lustre-OST0000-osc-ffff97ae0a7fe800.idle_timeout=0 osc.lustre-OST0001-osc-ffff97ae03c84000.idle_timeout=0 osc.lustre-OST0001-osc-ffff97ae0a7fe800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0396285 s, 207 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09448 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 283746 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0149729 s, 274 kB/s osc.lustre-OST0000-osc-ffff97ae03c84000.idle_timeout=debug osc.lustre-OST0000-osc-ffff97ae0a7fe800.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ae03c84000.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ae0a7fe800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 09:48:02 (1774532882) llite.lustre-ffff97ae03c84000.fast_read=0 llite.lustre-ffff97ae0a7fe800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00165132 s, 7.4 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff97ae03c84000.fast_read=1 llite.lustre-ffff97ae0a7fe800.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 09:48:09 (1774532889) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 09:48:10 (1774532890) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00149343 s, 2.7 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 09:48:12 (1774532892) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0233594 s, 89.8 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 09:48:14 (1774532894) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.021852 s, 96.0 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 09:48:16 (1774532896) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00125457 s, 9.6 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 09:48:18 (1774532898) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000474389 s, 8.6 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 09:48:20 (1774532900) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000446302 s, 9.2 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 09:48:22 (1774532902) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000413075 s, 9.9 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 09:48:26 (1774532906) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1774532906 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1774532909 Finishing test test23 at 1774532909 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 09:48:32 (1774532912) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 09:48:33 (1774532913) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 09:49:21 (1774532961) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 09:49:22 (1774532962) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00304513 s, 336 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:49:23.000000000 -0400 Modify: 2026-03-26 09:49:23.000000000 -0400 Change: 2026-03-26 09:49:23.000000000 -0400 Birth: 2026-03-26 09:49:22.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1774532962 1774532963 1774532963 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:49:23.000000000 -0400 Modify: 2026-03-26 09:49:23.000000000 -0400 Change: 2026-03-26 09:49:23.000000000 -0400 Birth: 2026-03-26 09:49:22.000000000 -0400 1774532963 1024 1 PASS 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 09:49:24 (1774532964) PASS 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 09:49:27 (1774532967) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0862222 s, 122 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2634 0xa4a 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 09:49:31 (1774532971) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0945208 s, 111 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2654 0xa5e 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7141 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 09:49:39 (1774532979) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00138697 s, 7.2 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.28811 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 09:49:45 (1774532985) 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre 192.168.202.160@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.160@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg260-client.virtnet /mnt/lustre (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: Force libcfs to create 2 CPU partitions Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre Starting client: oleg260-client.virtnet: -o user_xattr,flock oleg260-server@tcp:/lustre /mnt/lustre2 PASS 109 (529s) == sanityn test 110: do not grant another lock on resend ========================================================== 09:58:34 (1774533514) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-26 09:58:34.000000000 -0400 Modify: 2026-03-26 09:58:34.000000000 -0400 Change: 2026-03-26 09:58:34.000000000 -0400 Birth: 2026-03-26 09:58:34.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg260-client.virtnet oleg260-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 09:59:10 (1774533550) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 09:59:11 (1774533551) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 09:59:12 (1774533552) lustre-OST0000-osc-ffff97ae043da800 active. lustre-OST0001-osc-ffff97ae043da800 active. lustre-OST0000-osc-ffff97ae043da800 active. lustre-OST0001-osc-ffff97ae043da800 active. MGC192.168.202.160@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4862 sec ============== 09:59:15 (1774533555) Stopping clients: oleg260-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg260-client.virtnet /mnt/lustre2 opts: