-----============= acceptance-small: sanityn ============----- Fri Oct 24 09:11:37 EDT 2025 oleg405-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Started clients oleg405-client.virtnet: 192.168.204.105@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg405-client.virtnet: executing check_config_client /mnt/lustre oleg405-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg405-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9b4c870cd000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9b4c99097800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9b4c870cd000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9b4c99097800.idle_timeout=debug disable quota as required oleg405-server: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330537 s, 31.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7132] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 09:12:10 (1761311530) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (5s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 09:12:16 (1761311536) -rw-r--r-- 1 root root 0 Oct 24 09:12 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (6s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 09:12:21 (1761311541) -rw-r--r-- 1 root root 0 Oct 24 09:12 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (6s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 09:12:27 (1761311547) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (6s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 09:12:34 (1761311554) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (6s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 09:12:39 (1761311559) total 1 -rwxrwxrwx 1 root root 0 Oct 24 09:12 f2a -rwxrwxrwx 1 root root 0 Oct 24 09:12 f2b total 1 -rwxrwxrwx 1 root root 0 Oct 24 09:12 f2a -rwxrwxrwx 1 root root 0 Oct 24 09:12 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0151577 s, 33.8 kB/s PASS 2e (5s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 09:12:44 (1761311564) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (3s) == sanityn test 2g: check blocks update on sync write ==== 09:12:47 (1761311567) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.412351 s, 5.1 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (6s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 09:12:53 (1761311573) PASS 3 (5s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 09:12:58 (1761311578) PASS 4 (7s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 09:13:05 (1761311585) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (5s) == sanityn test 6: remove of open file on other node ============================================================================ 09:13:10 (1761311590) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (5s) == sanityn test 7: remove of open directory on other node ======================================================================= 09:13:16 (1761311596) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (6s) == sanityn test 8: remove of open special file on other node ==================================================================== 09:13:22 (1761311602) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (6s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 09:13:28 (1761311608) PASS 9a (7s) == sanityn test 9b: append to striped sparse file ======== 09:13:35 (1761311615) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038279 s, 27.4 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0710289 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (7s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 09:13:42 (1761311622) 1+0 records in 1+0 records out 1 byte copied, 0.00379741 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0134799 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00743277 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00416243 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00644563 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00436923 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00698208 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00645528 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00652795 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00426518 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00504766 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00418178 s, 0.2 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 09:13:49 (1761311629) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0484951 s, 63.3 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.04019 s, 102 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000275134 s, 11.2 MB/s PASS 10b (6s) == sanityn test 11: execution of file opened for write should return error ============================================================== 09:13:55 (1761311635) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (6s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 09:14:01 (1761311641) warning: '-runtime' deprecated, use '-t runtime' instead - link 428 (time 1761311653.17 total 10.02 last 42.69) - link 835 (time 1761311663.18 total 20.04 last 40.64) - link 1226 (time 1761311673.20 total 30.06 last 39.03) - link 1652 (time 1761311683.22 total 40.08 last 42.51) - link 2059 (time 1761311693.22 total 50.08 last 40.68) - link 2449 (time 1761311703.25 total 60.11 last 38.91) - link 2844 (time 1761311713.25 total 70.11 last 39.50) - link 3216 (time 1761311723.26 total 80.12 last 37.14) - link 3568 (time 1761311733.30 total 90.16 last 35.07) total: 3915 link in 100.01 seconds: 39.15 ops/second using seed 1631117059 running for 100 seconds - stat 10000 (time 1761311651 ; total 7 ; last 7) - stat 20000 (time 1761311659 ; total 15 ; last 8) - stat 30000 (time 1761311666 ; total 22 ; last 7) - stat 40000 (time 1761311674 ; total 30 ; last 8) - stat 50000 (time 1761311682 ; total 38 ; last 8) - stat 60000 (time 1761311689 ; total 45 ; last 7) - stat 69192 (time 1761311700 ; total 56 ; last 11) - stat 70000 (time 1761311701 ; total 57 ; last 1) - stat 77307 (time 1761311712 ; total 68 ; last 11) - stat 80000 (time 1761311716 ; total 72 ; last 4) - stat 87318 (time 1761311727 ; total 83 ; last 11) - stat 90000 (time 1761311731 ; total 87 ; last 4) - stat 96221 (time 1761311742 ; total 98 ; last 11) total: 97951 stats in 100 seconds: 979.510010 stats/second - unlinked 0 (time 1761311694 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile3915) error: No such file or directory total: 3914 unlinks in 86 seconds: 45.511627 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19704) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19706) - No such process PASS 12 (144s) == sanityn test 13: test directory page revocation ======= 09:16:25 (1761311785) f13 f13 PASS 13 (7s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 09:16:33 (1761311793) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20877 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (5s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 09:16:39 (1761311799) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21450 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (6s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 09:16:45 (1761311805) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22022 Terminated $DIR1/$tdir/sleep 60 PASS 14b (6s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 09:16:51 (1761311811) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22598 Terminated $DIR1/$tdir/sleep 60 PASS 14c (5s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 09:16:56 (1761311816) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23174 Terminated $DIR1/$tdir/sleep 60 PASS 14d (6s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 09:17:02 (1761311822) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (19s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 09:17:22 (1761311842) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.430661 s, 4.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.249134 s, 8.4 MB/s Chance of close/open is 1 in 50 Seed set to 1845 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1761311845.992288 mapwrite 0x3f34ed thru 0x3f5f26 (0x2a3a bytes) truncating to largest ever: 0x4c93ea 000003[0] 1761311846.249719 trunc from 0x3f5f27 to 0x4c93e9 (0xd34c3 bytes) 000005[0] 1761311846.309624 read 0x4b6a13 thru 0x4b70a4 (0x692 bytes) 000006[1] 1761311846.315754 read 0x3649ba thru 0x369b60 (0x51a7 bytes) 000009[0] 1761311846.344165 write 0x05a2bb thru 0x05dc9f (0x39e5 bytes) 000010[0] 1761311846.366237 trunc from 0x4c93ea to 0x2e53e6 (0x1e4003 bytes) 000011[0] 1761311846.419057 read 0x21fbb5 thru 0x2269ff (0x6e4b bytes) 000013[0] 1761311846.482261 read 0x2811a5 thru 0x28be47 (0xaca3 bytes) 000014[0] 1761311846.491468 read 0x2a7546 thru 0x2b2651 (0xb10c bytes) 000015[1] 1761311846.493848 trunc from 0x2e53e7 to 0x153750 (0x191c96 bytes) 000016[0] 1761311846.549928 mapread 0x0d6b24 thru 0x0e6810 (0xfced bytes) 000017[0] 1761311846.682935 write 0x544429 thru 0x54f33b (0xaf13 bytes) 000018[0] 1761311846.696239 trunc from 0x54f33c to 0x1365be (0x418d7d bytes) 000021[0] 1761311846.765048 write 0x01dfbf thru 0x02b2e1 (0xd323 bytes) 000023[0] 1761311846.829772 mapwrite 0x915b11 thru 0x9195a9 (0x3a99 bytes) 000025[0] 1761311847.048660 mapwrite 0x3341e4 thru 0x33ae75 (0x6c92 bytes) truncating to largest ever: 0x97d723 000026[1] 1761311847.140253 trunc from 0x9195aa to 0x97d722 (0x64179 bytes) 000028[1] 1761311847.195277 mapwrite 0x9d6b4f thru 0x9e3277 (0xc729 bytes) 000029[0] 1761311847.293951 write 0x0f794a thru 0x100833 (0x8eea bytes) 000030[0] 1761311847.331279 trunc from 0x9e3278 to 0x3dc0f2 (0x607185 bytes) 000031[0] 1761311847.353466 write 0x4a4054 thru 0x4adce0 (0x9c8d bytes) 000032[0] 1761311847.356286 mapread 0x036d5e thru 0x04563a (0xe8dd bytes) 000034[0] 1761311847.454249 read 0x1e8c2e thru 0x1f3e8c (0xb25f bytes) 000035[1] 1761311847.527254 read 0x0c2f5a thru 0x0d1f9f (0xf046 bytes) 000038[1] 1761311847.643083 mapread 0x0df5c0 thru 0x0e2971 (0x33b2 bytes) 000039[1] 1761311847.661721 read 0x1131ca thru 0x115456 (0x228d bytes) 000040[0] 1761311847.757836 mapread 0x203bca thru 0x21319d (0xf5d4 bytes) 000042[0] 1761311847.837639 read 0x2a3e29 thru 0x2a56ae (0x1886 bytes) 000043[1] 1761311847.874606 write 0x7aa7d4 thru 0x7ab17b (0x9a8 bytes) 000044[0] 1761311847.898960 write 0x16b19b thru 0x1707ca (0x5630 bytes) 000046[1] 1761311848.037543 trunc from 0x7ab17c to 0x23e0f8 (0x56d083 bytes) 000048[1] 1761311848.214698 read 0x09b31c thru 0x0aa971 (0xf656 bytes) 000050[0] 1761311848.256561 trunc from 0x23e0f9 to 0x86a7fe (0x62c706 bytes) 000052[0] 1761311848.340564 mapread 0x7329b4 thru 0x7424c8 (0xfb15 bytes) 000054[1] 1761311848.551983 mapwrite 0x0ce244 thru 0x0d6939 (0x86f6 bytes) 000055[1] 1761311848.759444 trunc from 0x86a7ff to 0x6be436 (0x1ac3c8 bytes) 000057[1] 1761311848.825793 mapwrite 0x957968 thru 0x962cff (0xb398 bytes) 000058[1] 1761311849.022407 write 0x86706d thru 0x86bea4 (0x4e38 bytes) 000059[0] 1761311849.023384 mapread 0x20d1e3 thru 0x2103fd (0x321b bytes) 000061[1] 1761311849.116716 read 0x7a5f5d thru 0x7aa756 (0x47fa bytes) 000062[0] 1761311849.163234 write 0x53f922 thru 0x5492d3 (0x99b2 bytes) 000064[1] 1761311849.240554 mapwrite 0x8c3300 thru 0x8cf2e6 (0xbfe7 bytes) 000065[0] 1761311849.399124 mapwrite 0x7bfa29 thru 0x7c13ec (0x19c4 bytes) 000066[1] 1761311849.479377 trunc from 0x962d00 to 0x69e5a1 (0x2c475e bytes) 000068[0] 1761311849.516422 trunc from 0x69e5a2 to 0x0be5d4 (0x5dffcd bytes) 000071[1] 1761311849.616163 read 0x03f397 thru 0x043602 (0x426c bytes) 000072[0] 1761311849.645452 mapwrite 0x145bd5 thru 0x14a225 (0x4651 bytes) 000073[1] 1761311849.752100 mapread 0x079f31 thru 0x082d9b (0x8e6b bytes) 000075[1] 1761311849.841147 mapread 0x11a2ab thru 0x12047a (0x61d0 bytes) 000076[1] 1761311849.869980 read 0x10d0f6 thru 0x1133b6 (0x62c1 bytes) 000077[0] 1761311849.877887 read 0x0b73ac thru 0x0b7d1d (0x972 bytes) truncating to largest ever: 0x9bce49 000080[1] 1761311849.896580 trunc from 0x14a226 to 0x9bce48 (0x872c23 bytes) 000082[1] 1761311849.980810 mapread 0x5eb526 thru 0x5ec033 (0xb0e bytes) 000083[0] 1761311850.162520 write 0x1d3271 thru 0x1dc9cf (0x975f bytes) 000085[1] 1761311850.239033 write 0x913cf0 thru 0x91c4bf (0x87d0 bytes) 000086[0] 1761311850.277029 trunc from 0x9bce49 to 0x93e022 (0x7ee26 bytes) 000088[1] 1761311850.398149 mapwrite 0x9515d4 thru 0x95cb14 (0xb541 bytes) 000089[0] 1761311850.549523 read 0x91726b thru 0x918c58 (0x19ee bytes) 000090[1] 1761311850.585508 mapread 0x0f1b83 thru 0x0f55cf (0x3a4d bytes) 000091[1] 1761311850.649125 write 0x438825 thru 0x43f001 (0x67dd bytes) 000092[0] 1761311850.700795 write 0x1529bf thru 0x15c661 (0x9ca3 bytes) 000095[1] 1761311850.838811 read 0x1e7147 thru 0x1ede52 (0x6d0c bytes) 000097[1] 1761311850.902534 write 0x11eb61 thru 0x12aa5b (0xbefb bytes) 000100[1] 1761311850.925073 mapwrite 0x2d2f87 thru 0x2de25d (0xb2d7 bytes) 000101[1] 1761311851.014629 mapwrite 0x9f1501 thru 0x9fc6a3 (0xb1a3 bytes) 000103[1] 1761311851.170750 read 0x00c041 thru 0x0192b9 (0xd279 bytes) 000104[1] 1761311851.280252 mapwrite 0x07a4f7 thru 0x085e1a (0xb924 bytes) 000106[1] 1761311851.390131 mapread 0x30f5fd thru 0x318ca8 (0x96ac bytes) 000109[1] 1761311851.417756 mapwrite 0x188c1e thru 0x18a3ae (0x1791 bytes) 000110[0] 1761311851.531582 mapwrite 0x0f609e thru 0x0f795d (0x18c0 bytes) 000111[1] 1761311851.699720 write 0x381c29 thru 0x38668b (0x4a63 bytes) 000113[1] 1761311851.732885 mapwrite 0x93cdff thru 0x946526 (0x9728 bytes) 000115[0] 1761311851.839900 read 0x4b1828 thru 0x4bcb55 (0xb32e bytes) 000116[0] 1761311851.878367 read 0x12eed4 thru 0x13ded3 (0xf000 bytes) 000119[0] 1761311852.039982 write 0x93502e thru 0x94219f (0xd172 bytes) 000121[1] 1761311852.072960 read 0x5c52ff thru 0x5d2610 (0xd312 bytes) 000122[0] 1761311852.130069 mapread 0x0ed1e8 thru 0x0ed539 (0x352 bytes) 000124[0] 1761311852.183993 trunc from 0x9fc6a4 to 0x80a1be (0x1f24e5 bytes) 000126[1] 1761311852.271520 write 0x9a05c6 thru 0x9abbb2 (0xb5ed bytes) 000129[0] 1761311852.426639 write 0x1ebcd6 thru 0x1ec24c (0x577 bytes) 000130[0] 1761311852.459274 read 0x0484ef thru 0x0505b2 (0x80c4 bytes) 000132[0] 1761311852.615742 mapwrite 0x87f609 thru 0x887756 (0x814e bytes) 000133[0] 1761311852.703776 mapread 0x8e0c0e thru 0x8e2918 (0x1d0b bytes) 000134[0] 1761311852.705269 mapread 0x05e4f8 thru 0x05f7db (0x12e4 bytes) 000135[1] 1761311852.705724 mapwrite 0x1b8973 thru 0x1c1133 (0x87c1 bytes) 000136[0] 1761311852.848147 mapread 0x157c14 thru 0x15ff2c (0x8319 bytes) 000137[0] 1761311852.914888 mapwrite 0x149970 thru 0x153a04 (0xa095 bytes) 000138[1] 1761311853.043297 write 0x3c65a2 thru 0x3d03d0 (0x9e2f bytes) 000140[1] 1761311853.112829 mapwrite 0x2bd400 thru 0x2c6792 (0x9393 bytes) 000142[0] 1761311853.383782 mapwrite 0x7c47c4 thru 0x7d35c3 (0xee00 bytes) 000144[1] 1761311853.576543 read 0x1ec4f2 thru 0x1ed67f (0x118e bytes) 000146[1] 1761311853.681730 mapwrite 0x09d75c thru 0x0ab2d2 (0xdb77 bytes) truncating to largest ever: 0x9d3763 000147[1] 1761311853.797154 trunc from 0x9abbb3 to 0x9d3762 (0x27bb0 bytes) 000148[0] 1761311853.828975 write 0x9eea03 thru 0x9eecde (0x2dc bytes) 000149[1] 1761311853.861895 read 0x2e015b thru 0x2e1b50 (0x19f6 bytes) 000150[0] 1761311853.881063 mapread 0x540b23 thru 0x54520e (0x46ec bytes) 000151[1] 1761311853.937583 mapwrite 0x3b2791 thru 0x3b7e8d (0x56fd bytes) 000152[1] 1761311854.034792 mapwrite 0x8cc2ab thru 0x8ccac4 (0x81a bytes) 000154[0] 1761311854.124319 trunc from 0x9eecdf to 0x3db67c (0x613662 bytes) 000155[1] 1761311854.194059 write 0x310032 thru 0x315e68 (0x5e37 bytes) 000156[0] 1761311854.259927 write 0x973433 thru 0x975477 (0x2045 bytes) 000158[1] 1761311854.349560 mapread 0x4a1c1c thru 0x4a73e1 (0x57c6 bytes) 000159[0] 1761311854.472145 trunc from 0x975478 to 0x454e16 (0x520661 bytes) 000160[0] 1761311854.532140 mapwrite 0x838d3f thru 0x843906 (0xabc8 bytes) 000161[0] 1761311854.781570 mapread 0x03022f thru 0x037208 (0x6fda bytes) 000162[0] 1761311854.854310 read 0x6a1af2 thru 0x6ac8fc (0xae0b bytes) 000163[0] 1761311854.954515 mapread 0x782f79 thru 0x78f5f9 (0xc681 bytes) 000164[1] 1761311854.955293 trunc from 0x843907 to 0x4d2f36 (0x3709d0 bytes) 000167[0] 1761311855.054760 mapwrite 0x9c8cc0 thru 0x9cafc0 (0x2301 bytes) 000168[0] 1761311855.156797 mapwrite 0x31b787 thru 0x321c24 (0x649e bytes) truncating to largest ever: 0x9e6a71 000169[0] 1761311855.295443 trunc from 0x9cafc1 to 0x9e6a70 (0x1bab0 bytes) 000170[0] 1761311855.316541 write 0x567a55 thru 0x56e8b9 (0x6e65 bytes) 000173[1] 1761311855.326019 read 0x75bece thru 0x75d061 (0x1194 bytes) 000175[1] 1761311855.382769 trunc from 0x9e6a71 to 0x57e46a (0x468606 bytes) 000176[1] 1761311855.456294 read 0x22c452 thru 0x235fef (0x9b9e bytes) 000177[0] 1761311855.524741 mapread 0x22fd73 thru 0x23d2b8 (0xd546 bytes) 000183[0] 1761311855.645987 mapwrite 0x726e12 thru 0x732624 (0xb813 bytes) 000185[0] 1761311855.826982 mapread 0x5a1cf2 thru 0x5a46e2 (0x29f1 bytes) 000186[0] 1761311855.873371 write 0x28886b thru 0x297602 (0xed98 bytes) 000188[0] 1761311855.905550 trunc from 0x732625 to 0x4ef5f8 (0x24302c bytes) 000191[0] 1761311855.994075 write 0x692818 thru 0x69df27 (0xb710 bytes) 000194[1] 1761311855.997372 write 0x3c672e thru 0x3c8d47 (0x261a bytes) 000196[1] 1761311856.057877 read 0x27530d thru 0x27a7d4 (0x54c8 bytes) 000197[0] 1761311856.167900 mapwrite 0x784553 thru 0x794409 (0xfeb7 bytes) 000199[1] 1761311856.379898 trunc from 0x79440a to 0x8d0d35 (0x13c92c bytes) 000200[1] 1761311856.481638 read 0x25ba17 thru 0x263d88 (0x8372 bytes) 000203[1] 1761311856.643973 mapread 0x0fb83e thru 0x103710 (0x7ed3 bytes) 000204[1] 1761311856.772643 mapwrite 0x340d63 thru 0x3420f3 (0x1391 bytes) 000206[1] 1761311856.817025 mapread 0x685c53 thru 0x690b79 (0xaf27 bytes) 000207[0] 1761311856.817726 mapread 0x687b6c thru 0x694170 (0xc605 bytes) 000208[0] 1761311856.932905 write 0x20990b thru 0x212df2 (0x94e8 bytes) 000209[0] 1761311856.967880 mapwrite 0x53417b thru 0x53c35e (0x81e4 bytes) 000210[1] 1761311857.089300 write 0x369aa8 thru 0x374af4 (0xb04d bytes) 000211[1] 1761311857.136145 mapwrite 0x593c6a thru 0x5a0d20 (0xd0b7 bytes) 000212[0] 1761311857.306561 read 0x6836b8 thru 0x685472 (0x1dbb bytes) 000213[1] 1761311857.350448 mapread 0x8a09f3 thru 0x8b000e (0xf61c bytes) 000214[1] 1761311857.556510 trunc from 0x8d0d36 to 0x7da726 (0xf660f bytes) 000215[1] 1761311857.590243 trunc from 0x7da727 to 0x7fa290 (0x1fb6a bytes) 000217[1] 1761311857.664557 mapread 0x3498b0 thru 0x35223a (0x898b bytes) 000218[1] 1761311857.755040 read 0x3c8b96 thru 0x3d0609 (0x7a74 bytes) 000222[1] 1761311857.911951 read 0x64f37b thru 0x6544d6 (0x515c bytes) 000223[0] 1761311857.912402 read 0x6cf4dc thru 0x6d8c69 (0x978e bytes) 000227[1] 1761311857.970619 mapread 0x61e6ae thru 0x621ae8 (0x343b bytes) 000228[1] 1761311858.001490 write 0x7e67e8 thru 0x7f1037 (0xa850 bytes) 000229[1] 1761311858.017953 read 0x2f15a4 thru 0x2f30b4 (0x1b11 bytes) 000232[1] 1761311858.065833 mapread 0x782f90 thru 0x78c9bf (0x9a30 bytes) 000233[1] 1761311858.069415 mapread 0x6b81c9 thru 0x6be269 (0x60a1 bytes) 000234[1] 1761311858.070686 mapwrite 0x9e5633 thru 0x9ed605 (0x7fd3 bytes) 000235[1] 1761311858.224133 mapwrite 0x151bac thru 0x15ea82 (0xced7 bytes) 000236[1] 1761311858.342569 write 0x86db06 thru 0x86f5ea (0x1ae5 bytes) 000240[0] 1761311858.417746 read 0x4199bf thru 0x41b99e (0x1fe0 bytes) 000241[0] 1761311858.455714 mapread 0x9d2d96 thru 0x9dc317 (0x9582 bytes) 000246[0] 1761311858.597752 write 0x5440ba thru 0x55284f (0xe796 bytes) 000247[1] 1761311858.667636 mapread 0x9938e0 thru 0x9a3101 (0xf822 bytes) 000249[0] 1761311858.784794 read 0x0800ba thru 0x081780 (0x16c7 bytes) 000251[1] 1761311858.860637 write 0x0a8ab2 thru 0x0a94da (0xa29 bytes) 000252[0] 1761311858.935428 mapwrite 0x87fb38 thru 0x887a4e (0x7f17 bytes) 000253[0] 1761311859.097882 trunc from 0x9ed606 to 0x5d53f7 (0x41820e bytes) 000255[1] 1761311859.184551 mapread 0x2b67a8 thru 0x2b7c2b (0x1484 bytes) 000256[1] 1761311859.244888 mapwrite 0x0757f7 thru 0x081634 (0xbe3e bytes) 000259[1] 1761311859.401564 mapwrite 0x9dd8a5 thru 0x9e4eda (0x7636 bytes) 000260[1] 1761311859.532830 mapread 0x37b4cb thru 0x38174b (0x6281 bytes) 000262[1] 1761311859.583326 mapwrite 0x2840d6 thru 0x2926ea (0xe615 bytes) 000264[1] 1761311859.725375 mapwrite 0x6cc68a thru 0x6d2fd9 (0x6950 bytes) 000265[0] 1761311859.819968 read 0x1e4acd thru 0x1e9597 (0x4acb bytes) 000267[1] 1761311859.845417 read 0x3785d5 thru 0x37c925 (0x4351 bytes) 000268[0] 1761311859.939305 write 0x7ac06e thru 0x7bb8a9 (0xf83c bytes) 000270[0] 1761311860.032301 mapread 0x742b33 thru 0x749329 (0x67f7 bytes) 000274[0] 1761311860.120843 trunc from 0x9e4edb to 0x88f737 (0x1557a3 bytes) 000275[1] 1761311860.175809 write 0x5206d1 thru 0x52c29e (0xbbce bytes) 000277[1] 1761311860.229281 read 0x410f1d thru 0x415dcd (0x4eb1 bytes) 000279[1] 1761311860.292689 write 0x831472 thru 0x83b6bc (0xa24b bytes) 000280[0] 1761311860.361949 read 0x556215 thru 0x5624ad (0xc299 bytes) 000281[0] 1761311860.424832 read 0x4bd731 thru 0x4be6a9 (0xf79 bytes) 000285[0] 1761311860.504136 mapread 0x546e02 thru 0x5515bc (0xa7bb bytes) 000286[1] 1761311860.514455 trunc from 0x88f738 to 0x55f75a (0x32ffdd bytes) 000287[0] 1761311860.625378 mapread 0x1eb83b thru 0x1ec52f (0xcf5 bytes) 000289[0] 1761311860.703400 mapwrite 0x8ac29f thru 0x8b4bb9 (0x891b bytes) 000290[1] 1761311860.813770 trunc from 0x8b4bba to 0x443a99 (0x471120 bytes) 000291[0] 1761311860.872696 trunc from 0x443a9a to 0x14977e (0x2fa31b bytes) 000294[0] 1761311860.909081 write 0x099fc9 thru 0x09d23c (0x3274 bytes) 000296[0] 1761311860.937676 mapread 0x13d8e9 thru 0x144e54 (0x756c bytes) 000298[1] 1761311860.995002 mapwrite 0x74293d thru 0x74f560 (0xcc24 bytes) 000302[0] 1761311861.210580 mapwrite 0x7d5aac thru 0x7d968a (0x3bdf bytes) 000303[0] 1761311861.343354 read 0x697f62 thru 0x69d733 (0x57d2 bytes) 000305[1] 1761311861.407283 mapwrite 0x840682 thru 0x844a35 (0x43b4 bytes) 000306[0] 1761311861.526722 mapread 0x3fcc60 thru 0x403ce1 (0x7082 bytes) 000308[1] 1761311861.650399 write 0x19b33d thru 0x1a4b54 (0x9818 bytes) 000310[0] 1761311861.693525 write 0x34d852 thru 0x35869a (0xae49 bytes) 000312[1] 1761311861.739617 read 0x338351 thru 0x3442db (0xbf8b bytes) 000316[1] 1761311861.837692 mapread 0x537946 thru 0x539256 (0x1911 bytes) 000317[1] 1761311861.839441 read 0x5971fb thru 0x59f175 (0x7f7b bytes) 000320[0] 1761311861.841765 write 0x767d19 thru 0x76d1c3 (0x54ab bytes) 000321[1] 1761311861.885468 write 0x9cd57a thru 0x9d37e0 (0x6267 bytes) 000322[1] 1761311861.912064 mapread 0x60f31b thru 0x613dbf (0x4aa5 bytes) 000324[1] 1761311861.925550 mapread 0x69a367 thru 0x69ed37 (0x49d1 bytes) 000325[1] 1761311861.936793 mapwrite 0x5eda84 thru 0x5f4fc4 (0x7541 bytes) 000327[1] 1761311862.019357 write 0x00c4c8 thru 0x01258d (0x60c6 bytes) 000328[0] 1761311862.043594 trunc from 0x9d37e1 to 0x08c4be (0x947322 bytes) 000332[0] 1761311862.153941 write 0x985de0 thru 0x990f29 (0xb14a bytes) 000333[1] 1761311862.179999 mapread 0x6dc0c5 thru 0x6e1830 (0x576c bytes) 000335[0] 1761311862.292374 mapwrite 0x00a3ea thru 0x0117b9 (0x73d0 bytes) 000336[1] 1761311862.426466 mapread 0x93aa29 thru 0x948f59 (0xe531 bytes) 000338[0] 1761311862.607953 mapread 0x98c624 thru 0x990f29 (0x4906 bytes) 000339[1] 1761311862.628590 mapread 0x6dbb81 thru 0x6e7048 (0xb4c8 bytes) 000340[0] 1761311862.756570 write 0x2edf2e thru 0x2f3a49 (0x5b1c bytes) 000341[1] 1761311862.829351 trunc from 0x990f2a to 0x8cb43b (0xc5aee bytes) 000343[0] 1761311862.917266 read 0x4de564 thru 0x4e8f62 (0xa9ff bytes) 000345[0] 1761311862.946792 write 0x57aca6 thru 0x588d7d (0xe0d8 bytes) 000348[1] 1761311863.023276 write 0x807aba thru 0x808a15 (0xf5c bytes) 000349[0] 1761311863.088823 write 0x05d39c thru 0x06843d (0xb0a2 bytes) 000352[1] 1761311863.136278 mapread 0x1389d9 thru 0x13c310 (0x3938 bytes) 000353[1] 1761311863.220443 read 0x85017a thru 0x85ab57 (0xa9de bytes) 000354[1] 1761311863.286639 write 0x1e233d thru 0x1eef85 (0xcc49 bytes) 000355[0] 1761311863.339475 mapwrite 0x617cca thru 0x61979a (0x1ad1 bytes) 000356[1] 1761311863.459589 trunc from 0x8cb43c to 0x4610a1 (0x46a39a bytes) 000358[0] 1761311863.497470 mapread 0x264944 thru 0x26768c (0x2d49 bytes) 000359[0] 1761311863.538323 read 0x058881 thru 0x064fa0 (0xc720 bytes) 000360[1] 1761311863.587761 read 0x3d2c4b thru 0x3d6afe (0x3eb4 bytes) 000362[0] 1761311863.650039 trunc from 0x4610a2 to 0x80955f (0x3a84be bytes) 000363[0] 1761311863.709084 mapread 0x6f8569 thru 0x705f2b (0xd9c3 bytes) 000364[1] 1761311863.880515 write 0x7613fb thru 0x764fef (0x3bf5 bytes) 000367[1] 1761311863.953718 mapread 0x3e8839 thru 0x3ec660 (0x3e28 bytes) 000368[1] 1761311864.001357 mapread 0x485e52 thru 0x48a430 (0x45df bytes) 000369[0] 1761311864.128124 write 0x8fbefd thru 0x9001f0 (0x42f4 bytes) 000370[1] 1761311864.185587 write 0x5a117d thru 0x5a13d1 (0x255 bytes) 000371[0] 1761311864.226888 write 0x7820b2 thru 0x782974 (0x8c3 bytes) 000376[1] 1761311864.314964 write 0x433ac8 thru 0x434721 (0xc5a bytes) 000377[0] 1761311864.405941 mapwrite 0x562997 thru 0x563d69 (0x13d3 bytes) 000378[1] 1761311864.518774 read 0x6459ff thru 0x64a002 (0x4604 bytes) 000379[1] 1761311864.551586 trunc from 0x9001f1 to 0x7d5e92 (0x12a35e bytes) 000380[1] 1761311864.592133 read 0x4ceb1e thru 0x4d8824 (0x9d07 bytes) 000382[1] 1761311864.672772 write 0x65ced8 thru 0x6611a9 (0x42d2 bytes) 000383[1] 1761311864.673595 mapwrite 0x99f66e thru 0x9aba50 (0xc3e3 bytes) 000384[0] 1761311864.800172 mapread 0x819252 thru 0x81e5cf (0x537e bytes) 000385[1] 1761311864.889400 write 0x7a05d7 thru 0x7a9c4a (0x9674 bytes) 000388[1] 1761311864.926407 mapwrite 0x3575c4 thru 0x35d0ba (0x5af7 bytes) 000389[0] 1761311865.027530 trunc from 0x9aba51 to 0x3e87f3 (0x5c325d bytes) 000390[1] 1761311865.081239 mapread 0x0f8752 thru 0x102d85 (0xa634 bytes) 000391[0] 1761311865.159498 read 0x237050 thru 0x2415e7 (0xa598 bytes) 000392[0] 1761311865.191349 read 0x0115ac thru 0x01917e (0x7bd3 bytes) 000395[1] 1761311865.250871 read 0x34e2f2 thru 0x351b14 (0x3823 bytes) 000397[1] 1761311865.268764 mapread 0x2a99cb thru 0x2aec53 (0x5289 bytes) 000398[0] 1761311865.297880 write 0x87ba3e thru 0x8812e8 (0x58ab bytes) 000399[1] 1761311865.334135 mapwrite 0x0fbd76 thru 0x10a52e (0xe7b9 bytes) 000400[0] 1761311865.663678 mapread 0x755b8e thru 0x75fcb3 (0xa126 bytes) 000401[1] 1761311865.810742 read 0x6232ef thru 0x630d0b (0xda1d bytes) 000402[1] 1761311865.826110 trunc from 0x8812e9 to 0x93a64c (0xb9364 bytes) 000403[1] 1761311865.865112 read 0x19afd7 thru 0x1a5a2c (0xaa56 bytes) 000406[1] 1761311865.993404 mapwrite 0x2a8f70 thru 0x2ad52d (0x45be bytes) 000407[0] 1761311866.036027 write 0x5ffacc thru 0x609a5e (0x9f93 bytes) 000408[0] 1761311866.124127 write 0x8a8449 thru 0x8b4c8a (0xc842 bytes) 000410[0] 1761311866.202039 mapwrite 0x15825f thru 0x15ecf1 (0x6a93 bytes) 000411[0] 1761311866.290746 write 0x3a694c thru 0x3ad4c9 (0x6b7e bytes) 000413[0] 1761311866.328816 mapwrite 0x181bfe thru 0x18d663 (0xba66 bytes) 000416[1] 1761311866.481337 mapwrite 0x898b39 thru 0x8a89f0 (0xfeb8 bytes) 000418[1] 1761311866.726680 trunc from 0x93a64d to 0x6e24b3 (0x258199 bytes) 000419[0] 1761311866.768240 trunc from 0x6e24b4 to 0x63a3be (0xa80f5 bytes) 000424[0] 1761311866.802962 mapread 0x62ad3c thru 0x639988 (0xec4d bytes) 000425[0] 1761311866.897593 read 0x0074f2 thru 0x00f26c (0x7d7b bytes) 000427[1] 1761311867.058338 write 0x76df02 thru 0x775278 (0x7377 bytes) 000428[0] 1761311867.093048 write 0x957cea thru 0x95ddd6 (0x60ed bytes) 000429[1] 1761311867.118493 trunc from 0x95ddd7 to 0x6b6f08 (0x2a6ece bytes) 000432[1] 1761311867.204591 trunc from 0x6b6f09 to 0x075aba (0x64144e bytes) 000434[0] 1761311867.250445 read 0x04a26a thru 0x04d92b (0x36c2 bytes) 000436[1] 1761311867.297689 write 0x71393f thru 0x71be97 (0x8559 bytes) 000437[1] 1761311867.300513 write 0x868d4a thru 0x86b0de (0x2395 bytes) 000439[1] 1761311867.314131 mapwrite 0x586904 thru 0x593dcc (0xd4c9 bytes) 000440[1] 1761311867.430363 trunc from 0x86b0df to 0x868cf1 (0x23ed bytes) 000441[1] 1761311867.473545 mapwrite 0x03386b thru 0x04039f (0xcb35 bytes) 000444[0] 1761311867.702178 trunc from 0x868cf2 to 0x597c33 (0x2d10be bytes) 000445[0] 1761311867.751792 mapwrite 0x1ad62f thru 0x1bbc9d (0xe66f bytes) 000447[1] 1761311867.975028 mapread 0x27a118 thru 0x2807fa (0x66e3 bytes) 000452[1] 1761311868.119018 mapread 0x51de33 thru 0x51f6de (0x18ac bytes) 000454[1] 1761311868.159056 write 0x8b0941 thru 0x8b2d42 (0x2402 bytes) 000455[0] 1761311868.189458 read 0x12f957 thru 0x13df75 (0xe61f bytes) 000456[0] 1761311868.226083 trunc from 0x8b2d43 to 0x0457e1 (0x86d561 bytes) 000458[0] 1761311868.324962 mapwrite 0x85b4d5 thru 0x85ed83 (0x38af bytes) 000459[0] 1761311868.615840 trunc from 0x85ed84 to 0x1b8765 (0x6a661e bytes) 000460[0] 1761311868.661586 mapwrite 0x8d5197 thru 0x8d5712 (0x57c bytes) 000461[1] 1761311868.736693 trunc from 0x8d5713 to 0x65a6da (0x27b038 bytes) 000462[0] 1761311868.798938 mapwrite 0x6fd4ab thru 0x70988f (0xc3e5 bytes) 000463[1] 1761311869.005982 write 0x97530d thru 0x97e09e (0x8d92 bytes) 000464[1] 1761311869.027860 trunc from 0x97e09f to 0x0f5ef7 (0x8881a7 bytes) 000465[0] 1761311869.071361 read 0x042d72 thru 0x049e37 (0x70c6 bytes) 000468[0] 1761311869.164535 write 0x177f02 thru 0x17e9b8 (0x6ab7 bytes) 000469[0] 1761311869.206053 trunc from 0x17e9b9 to 0x150cc1 (0x2dcf7 bytes) 000471[1] 1761311869.261154 trunc from 0x150cc2 to 0x020384 (0x13093d bytes) 000472[1] 1761311869.298085 mapread 0x01f2d4 thru 0x020384 (0x10b1 bytes) 000474[1] 1761311869.311785 write 0x5a6936 thru 0x5ae831 (0x7efc bytes) 000478[0] 1761311869.362791 mapread 0x48b207 thru 0x499b30 (0xe92a bytes) 000479[0] 1761311869.566456 mapread 0x410e50 thru 0x4127fd (0x19ae bytes) 000480[0] 1761311869.617113 mapwrite 0x8801ff thru 0x8811a5 (0xfa7 bytes) 000482[1] 1761311869.856461 read 0x84fd35 thru 0x85c969 (0xcc35 bytes) 000483[1] 1761311869.907180 mapwrite 0x74bc03 thru 0x74bf34 (0x332 bytes) 000484[0] 1761311869.974897 write 0x00d6a8 thru 0x01c431 (0xed8a bytes) 000487[0] 1761311870.046926 write 0x58b47b thru 0x590a59 (0x55df bytes) 000489[1] 1761311870.085245 mapwrite 0x77bffa thru 0x78640a (0xa411 bytes) 000490[1] 1761311870.201727 mapwrite 0x3dfd45 thru 0x3e682b (0x6ae7 bytes) 000493[1] 1761311870.330037 mapwrite 0x1a59ff thru 0x1a92c4 (0x38c6 bytes) 000494[0] 1761311870.435050 mapread 0x7865b4 thru 0x794990 (0xe3dd bytes) 000497[1] 1761311870.564999 write 0x090bcb thru 0x09d9bf (0xcdf5 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1870 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1761311871.283074 write 0x19c000 thru 0x19ced3 (0xed4 bytes) 000004[0] 1761311871.341240 mapwrite 0x3f4000 thru 0x403547 (0xf548 bytes) 000005[1] 1761311871.607432 mapwrite 0x948000 thru 0x94fd9a (0x7d9b bytes) 000009[1] 1761311871.779174 read 0x1d4000 thru 0x1ddd2b (0x9d2c bytes) 000010[1] 1761311871.807023 mapwrite 0x5f1000 thru 0x5fe485 (0xd486 bytes) 000011[1] 1761311871.903069 mapwrite 0x09f000 thru 0x0a4d95 (0x5d96 bytes) 000013[0] 1761311871.966534 mapread 0x04c000 thru 0x05a956 (0xe957 bytes) 000014[0] 1761311872.010566 read_OD 0x264000 thru 0x268fff (0x5000 bytes) 000015[1] 1761311872.021322 write 0x898000 thru 0x89bbe6 (0x3be7 bytes) 000016[1] 1761311872.046256 mapread 0x641000 thru 0x64de16 (0xce17 bytes) 000017[0] 1761311872.067157 read_OD 0x3d5000 thru 0x3defff (0xa000 bytes) 000018[0] 1761311872.093024 read_OD 0x7d0000 thru 0x7d0fff (0x1000 bytes) truncating to largest ever: 0x8e2d6 000019[1] 1761311872.103449 trunc from 0x94fd9b to 0x08e2d5 (0x8c1ac5 bytes) 000020[1] 1761311872.145154 mapread 0x043000 thru 0x04e76f (0xb770 bytes) 000021[0] 1761311872.176669 read_OD 0x072000 thru 0x07dfff (0xc000 bytes) 000022[1] 1761311872.203339 mapread 0x08c000 thru 0x08d45a (0x145b bytes) truncating to largest ever: 0x5ec426 000023[1] 1761311872.210724 trunc from 0x08e2d6 to 0x5ec425 (0x55e150 bytes) 000025[1] 1761311872.257126 trunc from 0x5ec426 to 0x50b262 (0xe11c3 bytes) 000028[0] 1761311872.288210 mapread 0x0e6000 thru 0x0edbde (0x7bdf bytes) 000030[0] 1761311872.376054 read_OD 0x155000 thru 0x162fff (0xe000 bytes) 000031[0] 1761311872.387693 read_OD 0x279000 thru 0x27dfff (0x5000 bytes) 000032[1] 1761311872.393540 mapwrite 0x305000 thru 0x311fe3 (0xcfe4 bytes) 000034[1] 1761311872.519440 read 0x1d6000 thru 0x1d7999 (0x199a bytes) 000036[1] 1761311872.593508 mapread 0x2a8000 thru 0x2ae0d7 (0x60d8 bytes) 000037[1] 1761311872.624848 write 0x7e0000 thru 0x7e202e (0x202f bytes) 000043[0] 1761311872.645928 mapwrite 0x8ab000 thru 0x8b57f3 (0xa7f4 bytes) 000047[0] 1761311872.914385 write_OD 0x2b9000 thru 0x2c1fff (0x9000 bytes) 000049[1] 1761311872.992092 trunc from 0x8b57f4 to 0x034cb2 (0x880b41 bytes) 000050[0] 1761311873.039782 mapread 0x00c000 thru 0x0165f0 (0xa5f1 bytes) truncating to largest ever: 0x903d41 000051[0] 1761311873.116077 trunc from 0x034cb3 to 0x903d40 (0x8cf08e bytes) 000052[1] 1761311873.146917 read 0x586000 thru 0x590255 (0xa256 bytes) 000056[1] 1761311873.225529 read 0x855000 thru 0x85d3db (0x83dc bytes) 000057[0] 1761311873.227229 read_OD 0x2d1000 thru 0x2dcfff (0xc000 bytes) 000058[0] 1761311873.233798 read_OD 0x361000 thru 0x36efff (0xe000 bytes) 000062[1] 1761311873.245951 trunc from 0x903d41 to 0x7ce91e (0x135422 bytes) 000063[1] 1761311873.326836 mapread 0x0fe000 thru 0x0ff701 (0x1702 bytes) 000064[0] 1761311873.359507 mapread 0x183000 thru 0x18cfc6 (0x9fc7 bytes) 000065[1] 1761311873.558603 mapwrite 0x76a000 thru 0x76c952 (0x2953 bytes) 000066[0] 1761311873.700957 write_OD 0x4fd000 thru 0x504fff (0x8000 bytes) 000067[1] 1761311873.744591 mapread 0x514000 thru 0x5165c8 (0x25c9 bytes) 000070[1] 1761311873.761704 read 0x75f000 thru 0x7665cb (0x75cc bytes) 000071[0] 1761311873.772900 mapread 0x416000 thru 0x418a51 (0x2a52 bytes) 000075[1] 1761311873.785934 mapwrite 0x5d7000 thru 0x5da209 (0x320a bytes) 000076[1] 1761311873.880401 mapwrite 0x061000 thru 0x0641fb (0x31fc bytes) 000079[1] 1761311873.987853 mapread 0x51e000 thru 0x525b12 (0x7b13 bytes) truncating to largest ever: 0x9fd1bb 000080[1] 1761311874.042316 trunc from 0x7ce91f to 0x9fd1ba (0x22e89c bytes) 000083[0] 1761311874.113384 write_OD 0x8e2000 thru 0x8ebfff (0xa000 bytes) 000085[0] 1761311874.164172 trunc from 0x9fd1bb to 0x505516 (0x4f7ca4 bytes) 000086[1] 1761311874.210404 mapwrite 0x282000 thru 0x2870f8 (0x50f9 bytes) 000087[1] 1761311874.299510 mapwrite 0x81a000 thru 0x81e063 (0x4064 bytes) 000088[0] 1761311874.370344 trunc from 0x81e064 to 0x66006e (0x1bdff5 bytes) 000089[0] 1761311874.406125 read_OD 0x35e000 thru 0x366fff (0x9000 bytes) 000090[1] 1761311874.423777 trunc from 0x66006f to 0x4b1f16 (0x1ae158 bytes) 000092[1] 1761311874.484153 trunc from 0x4b1f17 to 0x7be4a1 (0x30c58b bytes) 000095[1] 1761311874.574740 read 0x72e000 thru 0x73bf69 (0xdf6a bytes) 000097[0] 1761311874.629736 mapwrite 0x59c000 thru 0x5aab32 (0xeb33 bytes) 000098[0] 1761311874.790220 trunc from 0x7be4a2 to 0x1b734c (0x607155 bytes) 000099[1] 1761311874.823472 write 0x9b5000 thru 0x9c362b (0xe62c bytes) 000101[0] 1761311874.874097 write_OD 0x03a000 thru 0x045fff (0xc000 bytes) 000103[1] 1761311874.936881 mapread 0x96f000 thru 0x9711c6 (0x21c7 bytes) 000109[1] 1761311874.987372 write 0x1c0000 thru 0x1cc919 (0xc91a bytes) 000110[0] 1761311875.019426 read_OD 0x407000 thru 0x415fff (0xf000 bytes) 000111[0] 1761311875.046349 write_OD 0x1cc000 thru 0x1cefff (0x3000 bytes) 000114[1] 1761311875.094439 mapread 0x063000 thru 0x06532a (0x232b bytes) 000117[0] 1761311875.105366 read_OD 0x1df000 thru 0x1dffff (0x1000 bytes) 000120[1] 1761311875.110537 mapread 0x4cc000 thru 0x4dac88 (0xec89 bytes) 000121[0] 1761311875.177077 mapread 0x045000 thru 0x054806 (0xf807 bytes) 000123[0] 1761311875.370934 read_OD 0x763000 thru 0x764fff (0x2000 bytes) 000129[1] 1761311875.380635 mapwrite 0x904000 thru 0x905f4a (0x1f4b bytes) 000130[0] 1761311875.522655 read_OD 0x3cd000 thru 0x3d4fff (0x8000 bytes) 000132[1] 1761311875.544148 read 0x01f000 thru 0x02474f (0x5750 bytes) 000133[0] 1761311875.704459 read_OD 0x1e3000 thru 0x1ebfff (0x9000 bytes) 000134[0] 1761311875.752719 mapwrite 0x304000 thru 0x3072c8 (0x32c9 bytes) 000135[0] 1761311875.949417 trunc from 0x9c362c to 0x6b6f0c (0x30c71f bytes) 000136[1] 1761311875.970911 trunc from 0x6b6f0d to 0x72e949 (0x77a3d bytes) 000140[0] 1761311876.094997 trunc from 0x72e94a to 0x201067 (0x52d8e2 bytes) 000142[0] 1761311876.135982 mapread 0x134000 thru 0x13fa30 (0xba31 bytes) 000144[0] 1761311876.165062 mapread 0x044000 thru 0x053b2c (0xfb2d bytes) 000145[1] 1761311876.180456 trunc from 0x201068 to 0x10bbfc (0xf546b bytes) 000146[1] 1761311876.212948 mapread 0x0e2000 thru 0x0eace7 (0x8ce8 bytes) 000147[1] 1761311876.260390 mapwrite 0x7b5000 thru 0x7c4cdf (0xfce0 bytes) 000148[0] 1761311876.371497 trunc from 0x7c4ce0 to 0x1e0899 (0x5e4446 bytes) 000158[0] 1761311876.523745 mapread 0x06c000 thru 0x075a53 (0x9a54 bytes) 000160[0] 1761311876.557340 mapwrite 0x2fe000 thru 0x3031dd (0x51de bytes) 000161[1] 1761311876.656447 write 0x861000 thru 0x86f295 (0xe296 bytes) 000164[1] 1761311876.720835 read 0x300000 thru 0x30fded (0xfdee bytes) 000165[0] 1761311876.789390 mapread 0x3f2000 thru 0x3f62ef (0x42f0 bytes) 000166[0] 1761311876.887295 mapwrite 0x63a000 thru 0x644d10 (0xad11 bytes) 000168[0] 1761311877.072770 mapwrite 0x773000 thru 0x77a474 (0x7475 bytes) 000169[0] 1761311877.172922 mapwrite 0x746000 thru 0x74be0d (0x5e0e bytes) 000171[1] 1761311877.259735 read 0x5bf000 thru 0x5c10c8 (0x20c9 bytes) 000173[1] 1761311877.332258 trunc from 0x86f296 to 0x419717 (0x455b7e bytes) 000174[0] 1761311877.396098 write_OD 0x3da000 thru 0x3e6fff (0xd000 bytes) 000175[1] 1761311877.443996 mapread 0x11d000 thru 0x1227b3 (0x57b4 bytes) 000176[1] 1761311877.464505 write 0x816000 thru 0x81a108 (0x4109 bytes) 000178[1] 1761311877.509803 read 0x33c000 thru 0x33df6b (0x1f6c bytes) 000179[1] 1761311877.582858 mapwrite 0x249000 thru 0x257bcd (0xebce bytes) 000181[0] 1761311877.722141 mapread 0x1f5000 thru 0x1f7343 (0x2344 bytes) 000182[0] 1761311877.768057 write_OD 0x17e000 thru 0x181fff (0x4000 bytes) 000184[0] 1761311877.814921 trunc from 0x81a109 to 0x4252e6 (0x3f4e22 bytes) 000185[0] 1761311877.861225 read_OD 0x2c3000 thru 0x2c8fff (0x6000 bytes) 000186[0] 1761311877.878019 trunc from 0x4252e7 to 0x161297 (0x2c404f bytes) 000188[1] 1761311877.911924 read 0x119000 thru 0x11dcdf (0x4ce0 bytes) 000191[1] 1761311877.946175 write 0x6af000 thru 0x6bb942 (0xc943 bytes) 000192[1] 1761311877.996104 write 0x00a000 thru 0x019744 (0xf745 bytes) 000195[0] 1761311878.129294 write_OD 0x529000 thru 0x529fff (0x1000 bytes) 000196[0] 1761311878.218346 trunc from 0x6bb943 to 0x491396 (0x22a5ac bytes) 000197[0] 1761311878.275363 mapread 0x2f7000 thru 0x306f97 (0xff98 bytes) 000199[0] 1761311878.346896 read_OD 0x1dc000 thru 0x1e3fff (0x8000 bytes) 000201[0] 1761311878.371565 write_OD 0x374000 thru 0x378fff (0x5000 bytes) 000202[0] 1761311878.446918 mapwrite 0x714000 thru 0x7164b4 (0x24b5 bytes) 000205[1] 1761311878.654517 mapwrite 0x5ea000 thru 0x5f713a (0xd13b bytes) 000207[1] 1761311878.858218 mapread 0x19e000 thru 0x1a231f (0x4320 bytes) 000208[0] 1761311878.885076 mapread 0x19a000 thru 0x1a09c4 (0x69c5 bytes) 000209[1] 1761311878.986242 read 0x6e8000 thru 0x6f5301 (0xd302 bytes) 000213[1] 1761311878.995908 mapwrite 0x1eb000 thru 0x1f3135 (0x8136 bytes) 000215[1] 1761311879.139334 read 0x25b000 thru 0x25e578 (0x3579 bytes) 000217[1] 1761311879.273533 mapread 0x00b000 thru 0x015871 (0xa872 bytes) 000218[0] 1761311879.393740 read_OD 0x115000 thru 0x116fff (0x2000 bytes) 000219[1] 1761311879.445713 mapread 0x6c2000 thru 0x6cd34d (0xb34e bytes) 000221[0] 1761311879.508623 mapread 0x548000 thru 0x556c97 (0xec98 bytes) 000222[0] 1761311879.608921 read_OD 0x1e9000 thru 0x1f3fff (0xb000 bytes) 000226[0] 1761311879.618819 trunc from 0x7164b5 to 0x1ed657 (0x528e5d bytes) 000227[1] 1761311879.682844 trunc from 0x1ed658 to 0x764edf (0x577888 bytes) 000230[0] 1761311879.759600 mapwrite 0x279000 thru 0x27a412 (0x1413 bytes) 000232[0] 1761311880.023676 mapwrite 0x972000 thru 0x975afe (0x3aff bytes) 000233[1] 1761311880.166342 trunc from 0x975aff to 0x44027c (0x535882 bytes) 000234[1] 1761311880.221986 write 0x930000 thru 0x93b044 (0xb045 bytes) 000235[0] 1761311880.236833 write_OD 0x56b000 thru 0x571fff (0x7000 bytes) 000240[1] 1761311880.305493 mapread 0x56b000 thru 0x577005 (0xc006 bytes) 000243[0] 1761311880.410770 mapwrite 0x62a000 thru 0x636290 (0xc291 bytes) 000245[0] 1761311880.559657 read_OD 0x3ab000 thru 0x3b6fff (0xc000 bytes) 000247[0] 1761311880.610763 write 0x39c000 thru 0x3a8802 (0xc803 bytes) 000249[1] 1761311880.688466 write 0x36b000 thru 0x3772c6 (0xc2c7 bytes) 000250[0] 1761311880.721135 mapread 0x690000 thru 0x69bf91 (0xbf92 bytes) 000251[0] 1761311880.800583 write 0x34d000 thru 0x35a7d2 (0xd7d3 bytes) 000253[1] 1761311880.851028 mapwrite 0x416000 thru 0x418372 (0x2373 bytes) 000254[1] 1761311880.941831 mapwrite 0x3bd000 thru 0x3c60e8 (0x90e9 bytes) 000255[1] 1761311881.087409 write 0x8f7000 thru 0x8fb3b9 (0x43ba bytes) 000257[0] 1761311881.111394 mapwrite 0x474000 thru 0x47c513 (0x8514 bytes) 000259[1] 1761311881.240176 mapread 0x8ee000 thru 0x8fcfc2 (0xefc3 bytes) 000260[0] 1761311881.309586 read 0x7a6000 thru 0x7b36c5 (0xd6c6 bytes) 000263[1] 1761311881.318763 mapread 0x5da000 thru 0x5e4437 (0xa438 bytes) 000264[1] 1761311881.354929 mapread 0x0c6000 thru 0x0ce97f (0x8980 bytes) 000265[1] 1761311881.384590 mapwrite 0x231000 thru 0x23626e (0x526f bytes) 000266[0] 1761311881.462105 write 0x565000 thru 0x56b565 (0x6566 bytes) 000272[1] 1761311881.538652 trunc from 0x93b045 to 0x72abd2 (0x210472 bytes) 000273[0] 1761311881.635370 trunc from 0x72abd3 to 0x8217be (0xf6bec bytes) 000274[0] 1761311881.682532 read 0x6a7000 thru 0x6af20e (0x820f bytes) 000275[1] 1761311881.729806 write 0x3cd000 thru 0x3ce005 (0x1006 bytes) 000276[1] 1761311881.755012 write 0x5e4000 thru 0x5f07ff (0xc800 bytes) 000277[0] 1761311881.773034 write 0x94c000 thru 0x953257 (0x7258 bytes) 000278[1] 1761311881.819489 write 0x481000 thru 0x48f3aa (0xe3ab bytes) 000279[0] 1761311881.927268 mapread 0x7b2000 thru 0x7b3c1d (0x1c1e bytes) 000280[1] 1761311882.008043 write 0x7d0000 thru 0x7d83b8 (0x83b9 bytes) 000282[1] 1761311882.092999 write 0x137000 thru 0x1382e5 (0x12e6 bytes) 000283[0] 1761311882.099462 write 0x25f000 thru 0x26a489 (0xb48a bytes) 000284[0] 1761311882.165451 read 0x231000 thru 0x23517e (0x417f bytes) 000285[1] 1761311882.334814 mapwrite 0x3d0000 thru 0x3d350a (0x350b bytes) 000286[0] 1761311882.446551 trunc from 0x953258 to 0x746820 (0x20ca37 bytes) 000288[0] 1761311882.498928 read 0x51c000 thru 0x5286c6 (0xc6c7 bytes) 000289[1] 1761311882.576815 mapwrite 0x675000 thru 0x683cd2 (0xecd3 bytes) 000291[0] 1761311882.739314 mapread 0x053000 thru 0x05dea6 (0xaea7 bytes) 000292[0] 1761311882.856742 write 0x463000 thru 0x4645db (0x15dc bytes) 000294[1] 1761311882.931946 trunc from 0x746821 to 0x0f9899 (0x64cf87 bytes) 000295[0] 1761311882.994846 mapread 0x02a000 thru 0x02e3bf (0x43c0 bytes) 000296[0] 1761311883.057985 mapwrite 0x4ac000 thru 0x4b8362 (0xc363 bytes) 000298[0] 1761311883.270939 write 0x5ff000 thru 0x60d22b (0xe22c bytes) 000300[0] 1761311883.286837 mapread 0x1cd000 thru 0x1d4787 (0x7788 bytes) 000303[1] 1761311883.390720 write 0x5dd000 thru 0x5ddf06 (0xf07 bytes) 000305[1] 1761311883.451484 trunc from 0x60d22c to 0x21941c (0x3f3e0f bytes) 000307[0] 1761311883.511321 read 0x0f8000 thru 0x0fa124 (0x2125 bytes) 000308[1] 1761311883.562611 mapwrite 0x171000 thru 0x1743e9 (0x33ea bytes) 000310[0] 1761311883.633112 trunc from 0x21941d to 0x887882 (0x66e466 bytes) 000312[1] 1761311883.678414 mapwrite 0x239000 thru 0x248fa3 (0xffa4 bytes) 000314[1] 1761311883.820159 trunc from 0x887883 to 0x88b976 (0x40f4 bytes) 000315[0] 1761311883.854333 mapread 0x3df000 thru 0x3e759a (0x859b bytes) 000319[1] 1761311883.963978 mapread 0x065000 thru 0x06b48f (0x6490 bytes) 000321[0] 1761311884.001377 mapread 0x116000 thru 0x1190a1 (0x30a2 bytes) 000323[1] 1761311884.018766 mapread 0x6f2000 thru 0x6f5bfe (0x3bff bytes) 000327[0] 1761311884.038817 mapread 0x10d000 thru 0x119829 (0xc82a bytes) 000328[0] 1761311884.081855 read 0x056000 thru 0x05ea72 (0x8a73 bytes) 000331[0] 1761311884.121062 read 0x59c000 thru 0x5a2dec (0x6ded bytes) 000334[0] 1761311884.121972 trunc from 0x88b977 to 0x02c280 (0x85f6f6 bytes) 000335[0] 1761311884.192308 mapread 0x009000 thru 0x014c28 (0xbc29 bytes) 000336[0] 1761311884.261669 mapwrite 0x2ec000 thru 0x2f61f6 (0xa1f7 bytes) 000338[1] 1761311884.410884 trunc from 0x2f61f7 to 0x39481f (0x9e629 bytes) 000341[1] 1761311884.473994 read 0x369000 thru 0x370e95 (0x7e96 bytes) 000342[0] 1761311884.494998 read 0x167000 thru 0x170b9f (0x9ba0 bytes) 000343[1] 1761311884.600795 mapwrite 0x35b000 thru 0x3650d0 (0xa0d1 bytes) 000344[1] 1761311884.791052 trunc from 0x394820 to 0x85a768 (0x4c5f49 bytes) 000346[0] 1761311884.817290 read 0x07c000 thru 0x07db77 (0x1b78 bytes) 000347[1] 1761311884.967835 trunc from 0x85a769 to 0x26b9df (0x5eed89 bytes) 000350[0] 1761311885.044913 read 0x182000 thru 0x187847 (0x5848 bytes) 000351[0] 1761311885.086535 write 0x098000 thru 0x09f4ea (0x74eb bytes) 000356[1] 1761311885.151015 read 0x15e000 thru 0x169c8b (0xbc8c bytes) 000359[1] 1761311885.222321 read 0x12e000 thru 0x12f7dc (0x17dd bytes) 000360[0] 1761311885.240318 mapwrite 0x89e000 thru 0x8ac1d4 (0xe1d5 bytes) 000361[1] 1761311885.451452 write 0x82d000 thru 0x830ddc (0x3ddd bytes) 000363[0] 1761311885.553108 write 0x3a6000 thru 0x3a970c (0x370d bytes) 000364[1] 1761311885.596342 write 0x3c2000 thru 0x3c88a5 (0x68a6 bytes) 000366[1] 1761311885.627821 trunc from 0x8ac1d5 to 0x8c4eb1 (0x18cdd bytes) 000367[0] 1761311885.645268 mapread 0x26e000 thru 0x270320 (0x2321 bytes) 000368[0] 1761311885.696096 trunc from 0x8c4eb2 to 0x1ea8b0 (0x6da601 bytes) 000369[0] 1761311885.749826 write 0x4cc000 thru 0x4cee4f (0x2e50 bytes) 000371[0] 1761311885.767979 trunc from 0x4cee50 to 0x49963e (0x35811 bytes) 000374[1] 1761311885.818001 read 0x2dc000 thru 0x2df066 (0x3067 bytes) 000377[1] 1761311885.870971 read 0x414000 thru 0x415e57 (0x1e58 bytes) 000378[0] 1761311885.871368 trunc from 0x49963f to 0x26e20a (0x22b434 bytes) 000379[1] 1761311885.911505 write 0x50d000 thru 0x519a2d (0xca2e bytes) 000382[0] 1761311885.963792 write 0x026000 thru 0x0289b3 (0x29b4 bytes) 000384[1] 1761311886.024936 mapread 0x090000 thru 0x096f90 (0x6f91 bytes) 000388[1] 1761311886.077088 read 0x0af000 thru 0x0b9614 (0xa615 bytes) 000391[0] 1761311886.117621 mapwrite 0x407000 thru 0x40d00d (0x600e bytes) 000392[1] 1761311886.232551 mapwrite 0x085000 thru 0x08b4f7 (0x64f8 bytes) 000393[1] 1761311886.360717 write 0x747000 thru 0x756320 (0xf321 bytes) 000394[1] 1761311886.375642 write 0x622000 thru 0x631594 (0xf595 bytes) 000397[0] 1761311886.416360 mapwrite 0x62b000 thru 0x631697 (0x6698 bytes) 000399[0] 1761311886.524093 mapread 0x3f4000 thru 0x3fc01d (0x801e bytes) 000400[1] 1761311886.579706 mapwrite 0x4cd000 thru 0x4d4072 (0x7073 bytes) 000402[1] 1761311886.700104 trunc from 0x756321 to 0x3531b1 (0x40316f bytes) 000404[1] 1761311886.734631 mapread 0x12b000 thru 0x137743 (0xc744 bytes) 000405[0] 1761311886.798990 trunc from 0x3531b2 to 0x0dfbe9 (0x2735c8 bytes) 000407[0] 1761311886.840929 read 0x04e000 thru 0x05b9da (0xd9db bytes) 000410[0] 1761311886.916311 read 0x0c8000 thru 0x0cc273 (0x4274 bytes) 000411[0] 1761311886.923816 trunc from 0x0dfbea to 0x9ce3ac (0x8ee7c3 bytes) 000412[1] 1761311886.967843 trunc from 0x9ce3ad to 0x64e578 (0x37fe34 bytes) 000413[0] 1761311887.053187 mapwrite 0x7fd000 thru 0x80044e (0x344f bytes) 000414[0] 1761311887.171314 trunc from 0x80044f to 0x8753b4 (0x74f66 bytes) 000415[0] 1761311887.182510 trunc from 0x8753b5 to 0x263c38 (0x61177c bytes) 000416[1] 1761311887.203921 trunc from 0x263c39 to 0x12a7e4 (0x139454 bytes) 000418[0] 1761311887.221308 mapwrite 0x02e000 thru 0x03db28 (0xfb29 bytes) 000419[0] 1761311887.359036 read 0x049000 thru 0x0492f2 (0x2f3 bytes) 000420[0] 1761311887.391162 mapread 0x005000 thru 0x005e48 (0xe49 bytes) 000421[0] 1761311887.419973 mapwrite 0x823000 thru 0x829735 (0x6736 bytes) 000422[1] 1761311887.599540 mapread 0x4dd000 thru 0x4dd58f (0x590 bytes) 000424[0] 1761311887.687983 write 0x304000 thru 0x309b99 (0x5b9a bytes) 000426[1] 1761311887.725785 read 0x339000 thru 0x33bb09 (0x2b0a bytes) 000427[1] 1761311887.847123 trunc from 0x829736 to 0x31ec87 (0x50aaae bytes) 000428[1] 1761311887.888384 mapwrite 0x797000 thru 0x7a2815 (0xb816 bytes) 000429[1] 1761311887.996230 mapwrite 0x3e8000 thru 0x3f3ee3 (0xbee4 bytes) 000433[0] 1761311888.141340 trunc from 0x7a2816 to 0x3c8877 (0x3d9f9e bytes) 000434[0] 1761311888.181028 mapwrite 0x254000 thru 0x2599c8 (0x59c9 bytes) 000435[1] 1761311888.264380 mapread 0x25a000 thru 0x25f225 (0x5226 bytes) 000438[1] 1761311888.337617 read 0x380000 thru 0x389d59 (0x9d5a bytes) 000441[0] 1761311888.351952 trunc from 0x3c8878 to 0x66fd3a (0x2a74c3 bytes) 000443[0] 1761311888.399090 mapwrite 0x931000 thru 0x9362e4 (0x52e5 bytes) 000445[0] 1761311888.576995 write 0x7b8000 thru 0x7be755 (0x6756 bytes) 000446[1] 1761311888.606406 trunc from 0x9362e5 to 0x68a30d (0x2abfd7 bytes) 000447[1] 1761311888.648119 read 0x158000 thru 0x15ea04 (0x6a05 bytes) 000448[1] 1761311888.749657 write 0x006000 thru 0x008cc3 (0x2cc4 bytes) 000449[1] 1761311888.758901 mapread 0x0e2000 thru 0x0eb737 (0x9738 bytes) 000450[1] 1761311888.801147 mapread 0x4ee000 thru 0x4f1b4b (0x3b4c bytes) 000453[1] 1761311888.816457 read 0x32d000 thru 0x3353e9 (0x83ea bytes) 000454[0] 1761311888.843889 mapread 0x54b000 thru 0x55ae39 (0xfe3a bytes) 000457[0] 1761311889.023593 mapread 0x29a000 thru 0x2a7ce6 (0xdce7 bytes) 000459[1] 1761311889.100397 mapread 0x03d000 thru 0x03f812 (0x2813 bytes) 000462[0] 1761311889.119718 mapwrite 0x846000 thru 0x85141e (0xb41f bytes) 000465[1] 1761311889.325153 write 0x74a000 thru 0x74d111 (0x3112 bytes) 000467[0] 1761311889.364339 mapwrite 0x78d000 thru 0x795b45 (0x8b46 bytes) 000469[0] 1761311889.553715 mapwrite 0x934000 thru 0x93c31c (0x831d bytes) 000476[1] 1761311889.811854 trunc from 0x93c31d to 0x2b2e27 (0x6894f5 bytes) 000478[0] 1761311889.866955 mapread 0x1b0000 thru 0x1b5198 (0x5199 bytes) 000479[0] 1761311889.908461 trunc from 0x2b2e28 to 0x4cd68b (0x21a864 bytes) 000480[0] 1761311889.976143 trunc from 0x4cd68c to 0x4cf19c (0x1b11 bytes) 000486[0] 1761311890.112762 write 0x0bc000 thru 0x0c0954 (0x4955 bytes) 000492[1] 1761311890.214350 write 0x6c0000 thru 0x6c3781 (0x3782 bytes) 000494[1] 1761311890.247262 read 0x0be000 thru 0x0cd24e (0xf24f bytes) 000495[1] 1761311890.336349 mapwrite 0x028000 thru 0x033f25 (0xbf26 bytes) 000497[1] 1761311890.488592 write 0x679000 thru 0x67cee9 (0x3eea bytes) 000498[1] 1761311890.510035 mapread 0x0e3000 thru 0x0ee7c4 (0xb7c5 bytes) All operations completed A-OK! PASS 16a (53s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 09:18:15 (1761311895) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.316064 s, 6.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.291879 s, 7.2 MB/s Chance of close/open is 1 in 50 Seed set to 1897 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x6cf49e 000001[0] 1761311898.277470 trunc from 00000000 to 0x6cf49d (0x6cf49e bytes) 000005[0] 1761311898.295050 mapread 0x686358 thru 0x6880db (0x1d84 bytes) 000006[0] 1761311898.334849 mapread 0x1f0a10 thru 0x1f211b (0x170c bytes) 000009[1] 1761311898.429395 write 0x211429 thru 0x211b4f (0x727 bytes) 000010[0] 1761311898.459147 mapread 0x33fb45 thru 0x34067e (0xb3a bytes) 000011[1] 1761311898.487995 mapread 0x333847 thru 0x334af5 (0x12af bytes) 000012[1] 1761311898.536944 read 0x35d118 thru 0x35ec05 (0x1aee bytes) 000014[1] 1761311898.616433 read 0x27b26d thru 0x27bec8 (0xc5c bytes) 000017[1] 1761311898.760286 trunc from 0x6cf49e to 0x2d689e (0x3f8bff bytes) 000018[1] 1761311898.876641 write 0x26f048 thru 0x270cc1 (0x1c7a bytes) 000019[0] 1761311898.914811 mapwrite 0x4a541b thru 0x4a633c (0xf22 bytes) 000020[0] 1761311899.122281 trunc from 0x4a633d to 0x3d8067 (0xce2d5 bytes) 000022[1] 1761311899.164321 write 0x62a49c thru 0x62ae51 (0x9b6 bytes) 000023[1] 1761311899.185876 mapread 0x1a91cd thru 0x1a996a (0x79e bytes) 000026[0] 1761311899.221466 write 0x485ecd thru 0x4860d3 (0x207 bytes) 000027[0] 1761311899.294782 write 0x215499 thru 0x2171a9 (0x1d11 bytes) 000028[1] 1761311899.327593 mapread 0x4b0ea6 thru 0x4b2697 (0x17f2 bytes) 000029[0] 1761311899.367840 write 0x813058 thru 0x813464 (0x40d bytes) 000031[1] 1761311899.410747 read 0x4df10e thru 0x4e0d36 (0x1c29 bytes) truncating to largest ever: 0x8b9074 000032[1] 1761311899.523243 trunc from 0x813465 to 0x8b9073 (0xa5c0f bytes) truncating to largest ever: 0x8f5b51 000033[0] 1761311899.598867 trunc from 0x8b9074 to 0x8f5b50 (0x3cadd bytes) 000035[1] 1761311899.702295 read 0x4e5f74 thru 0x4e6c41 (0xcce bytes) 000038[0] 1761311899.761003 trunc from 0x8f5b51 to 0x539bcc (0x3bbf84 bytes) 000040[0] 1761311899.833815 mapwrite 0x95f75f thru 0x961113 (0x19b5 bytes) 000041[1] 1761311899.977717 mapwrite 0x0d9b99 thru 0x0da0bf (0x527 bytes) 000042[0] 1761311900.075295 trunc from 0x961114 to 0x721906 (0x23f80d bytes) 000047[0] 1761311900.104596 mapwrite 0x06120d thru 0x062565 (0x1359 bytes) 000048[1] 1761311900.154926 mapread 0x105871 thru 0x107792 (0x1f22 bytes) 000051[1] 1761311900.268125 write 0x8dbcd5 thru 0x8dd372 (0x169e bytes) 000053[0] 1761311900.345961 mapread 0x73200b thru 0x733858 (0x184e bytes) 000054[0] 1761311900.402155 trunc from 0x8dd373 to 0x0c677e (0x816bf4 bytes) 000055[0] 1761311900.451975 mapwrite 0x50b117 thru 0x50c1a9 (0x1093 bytes) 000056[0] 1761311900.551103 write 0x98acd8 thru 0x98b0a0 (0x3c9 bytes) 000057[0] 1761311900.569961 mapread 0x4ff289 thru 0x500f91 (0x1d09 bytes) 000060[1] 1761311900.606510 mapread 0x487056 thru 0x487b8c (0xb37 bytes) 000061[0] 1761311900.633994 read 0x6a976a thru 0x6a9a99 (0x330 bytes) 000065[0] 1761311900.693254 read 0x396ecf thru 0x398973 (0x1aa5 bytes) 000066[1] 1761311900.794646 write 0x3551db thru 0x357072 (0x1e98 bytes) 000067[1] 1761311900.917472 read 0x6b6ae7 thru 0x6b87cc (0x1ce6 bytes) 000070[1] 1761311900.944368 read 0x558b0d thru 0x55a915 (0x1e09 bytes) 000072[1] 1761311900.978399 mapwrite 0x309c8c thru 0x30b92d (0x1ca2 bytes) 000073[0] 1761311901.053045 trunc from 0x98b0a1 to 0x709a92 (0x28160e bytes) 000075[1] 1761311901.144984 mapread 0x0bf0c1 thru 0x0bf7f6 (0x736 bytes) 000076[1] 1761311901.210170 trunc from 0x709a93 to 0x8cb553 (0x1c1ac1 bytes) 000077[0] 1761311901.314100 trunc from 0x8cb554 to 0x047aa5 (0x883aae bytes) 000078[0] 1761311901.410184 mapwrite 0x000a94 thru 0x0010d9 (0x646 bytes) 000081[1] 1761311901.521117 trunc from 0x047aa6 to 0x845059 (0x7fd5b4 bytes) 000082[0] 1761311901.625422 read 0x49264e thru 0x492af1 (0x4a4 bytes) 000084[0] 1761311901.688355 read 0x5c7243 thru 0x5c778d (0x54b bytes) 000085[0] 1761311901.766711 trunc from 0x84505a to 0x0d9792 (0x76b8c7 bytes) 000088[0] 1761311901.885733 trunc from 0x0d9793 to 0x62c4a5 (0x552d13 bytes) 000091[0] 1761311901.975482 read 0x40918c thru 0x409ca6 (0xb1b bytes) 000092[1] 1761311902.069102 mapwrite 0x489fcd thru 0x48adfa (0xe2e bytes) 000093[0] 1761311902.201172 trunc from 0x62c4a6 to 0x1f77a4 (0x434d01 bytes) 000094[0] 1761311902.249490 write 0x6c980b thru 0x6ca000 (0x7f6 bytes) 000095[1] 1761311902.268817 mapwrite 0x8d689f thru 0x8d7386 (0xae8 bytes) 000098[1] 1761311902.404346 write 0x8eeb76 thru 0x8efed5 (0x1360 bytes) 000099[1] 1761311902.436487 mapread 0x2808fb thru 0x281a15 (0x111b bytes) truncating to largest ever: 0x93c774 000100[0] 1761311902.445170 trunc from 0x8efed6 to 0x93c773 (0x4c89e bytes) 000101[0] 1761311902.504432 trunc from 0x93c774 to 0x04df3b (0x8ee838 bytes) 000102[1] 1761311902.556404 write 0x0553b3 thru 0x0561d6 (0xe24 bytes) 000103[1] 1761311902.585587 trunc from 0x0561d7 to 0x323296 (0x2cd0c0 bytes) 000104[1] 1761311902.619694 mapread 0x0b0997 thru 0x0b0ba5 (0x20f bytes) 000106[0] 1761311902.670880 read 0x123530 thru 0x123962 (0x433 bytes) 000107[0] 1761311902.713814 trunc from 0x323297 to 0x7f2456 (0x4cf1c0 bytes) 000108[0] 1761311902.795931 read 0x270080 thru 0x271afb (0x1a7c bytes) 000110[0] 1761311902.969231 mapread 0x3a21e2 thru 0x3a26c6 (0x4e5 bytes) 000111[0] 1761311902.994539 mapread 0x09fef1 thru 0x0a128a (0x139a bytes) 000113[0] 1761311903.048331 mapwrite 0x60dc9b thru 0x60f1cd (0x1533 bytes) 000114[0] 1761311903.120071 trunc from 0x7f2457 to 0x0c32e8 (0x72f16e bytes) 000115[0] 1761311903.161847 read 0x0706b9 thru 0x0713ef (0xd37 bytes) 000116[1] 1761311903.208180 read 0x09eb98 thru 0x0a0262 (0x16cb bytes) 000117[0] 1761311903.232060 mapread 0x03200a thru 0x032e68 (0xe5f bytes) 000118[1] 1761311903.238852 write 0x3d417c thru 0x3d4df7 (0xc7c bytes) 000119[1] 1761311903.250831 read 0x08fcd9 thru 0x090058 (0x380 bytes) 000125[1] 1761311903.344906 write 0x89cb2a thru 0x89eb02 (0x1fd9 bytes) 000126[0] 1761311903.388674 trunc from 0x89eb03 to 0x219796 (0x68536c bytes) 000130[0] 1761311903.517446 mapwrite 0x6ad0db thru 0x6ad71a (0x640 bytes) 000131[0] 1761311903.690151 mapread 0x16b424 thru 0x16b5f0 (0x1cd bytes) 000132[1] 1761311903.690704 write 0x3e035d thru 0x3e1741 (0x13e5 bytes) 000133[0] 1761311903.725781 mapread 0x557d33 thru 0x558f71 (0x123f bytes) 000135[0] 1761311903.790082 mapwrite 0x2e7e50 thru 0x2e9a00 (0x1bb1 bytes) 000136[0] 1761311903.837095 mapread 0x5585e6 thru 0x55a03b (0x1a56 bytes) 000144[1] 1761311903.902430 mapwrite 0x25efd7 thru 0x25f219 (0x243 bytes) 000145[1] 1761311904.031143 trunc from 0x6ad71b to 0x59fe37 (0x10d8e3 bytes) 000146[0] 1761311904.076954 trunc from 0x59fe38 to 0x55a5e9 (0x4584e bytes) 000148[1] 1761311904.213937 write 0x0a6d1e thru 0x0a6e86 (0x169 bytes) 000151[1] 1761311904.307375 mapread 0x27792c thru 0x277dc8 (0x49d bytes) 000153[0] 1761311904.360918 read 0x3ad839 thru 0x3ae755 (0xf1d bytes) truncating to largest ever: 0x9aa44a 000154[1] 1761311904.403040 trunc from 0x55a5ea to 0x9aa449 (0x44fe60 bytes) 000155[1] 1761311904.441789 mapwrite 0x227589 thru 0x229160 (0x1bd8 bytes) 000156[0] 1761311904.533662 mapwrite 0x3f6f40 thru 0x3f8276 (0x1337 bytes) 000159[1] 1761311904.748082 read 0x8e8cc6 thru 0x8eabf3 (0x1f2e bytes) 000160[1] 1761311904.767391 trunc from 0x9aa44a to 0x33cc46 (0x66d803 bytes) 000162[1] 1761311904.840363 mapwrite 0x72dacd thru 0x72e9c8 (0xefc bytes) 000163[1] 1761311904.940921 trunc from 0x72e9c9 to 0x176e9d (0x5b7b2b bytes) 000164[1] 1761311904.959772 mapread 0x106a45 thru 0x10718c (0x748 bytes) 000165[1] 1761311904.990167 trunc from 0x176e9e to 0x7a1d11 (0x62ae74 bytes) 000167[1] 1761311905.024667 write 0x255929 thru 0x25770d (0x1de5 bytes) 000169[1] 1761311905.115896 trunc from 0x7a1d12 to 0x0f65d4 (0x6ab73d bytes) 000174[1] 1761311905.170790 mapread 0x0ba944 thru 0x0bb5c5 (0xc82 bytes) 000177[0] 1761311905.209727 mapwrite 0x227fd2 thru 0x228db2 (0xde1 bytes) 000178[1] 1761311905.391544 mapwrite 0x2ad41b thru 0x2ae072 (0xc58 bytes) 000179[0] 1761311905.466326 read 0x061fd3 thru 0x0638cc (0x18fa bytes) 000181[1] 1761311905.529100 write 0x05775d thru 0x059572 (0x1e16 bytes) 000183[1] 1761311905.588022 write 0x5ff0bf thru 0x601029 (0x1f6b bytes) 000186[1] 1761311905.668012 mapread 0x2960ea thru 0x2973e3 (0x12fa bytes) 000188[0] 1761311905.720846 trunc from 0x60102a to 0x42c5ca (0x1d4a5f bytes) 000189[0] 1761311905.806386 mapread 0x3ffdc2 thru 0x400099 (0x2d8 bytes) 000190[0] 1761311905.892187 read 0x276e08 thru 0x278698 (0x1891 bytes) 000191[0] 1761311905.892644 mapread 0x3d48cf thru 0x3d67f4 (0x1f26 bytes) 000194[0] 1761311905.936179 mapwrite 0x74714c thru 0x7482b4 (0x1169 bytes) 000196[0] 1761311906.004352 mapread 0x45de71 thru 0x45e3fd (0x58d bytes) 000197[0] 1761311906.052883 mapwrite 0x177e58 thru 0x179d9b (0x1f44 bytes) 000198[0] 1761311906.122777 trunc from 0x7482b5 to 0x15b615 (0x5ecc9f bytes) 000199[1] 1761311906.192694 write 0x775191 thru 0x77704f (0x1ebf bytes) 000202[0] 1761311906.295587 trunc from 0x777050 to 0x341a82 (0x4355cd bytes) 000209[0] 1761311906.408292 mapread 0x14984d thru 0x14adb1 (0x1565 bytes) 000213[1] 1761311906.471019 trunc from 0x341a83 to 0x625a33 (0x2e3fb1 bytes) 000214[0] 1761311906.510374 write 0x64e258 thru 0x64f613 (0x13bc bytes) 000217[0] 1761311906.566023 trunc from 0x64f614 to 0x31640b (0x339208 bytes) 000219[1] 1761311906.622840 mapwrite 0x75127b thru 0x751ef5 (0xc7b bytes) 000220[1] 1761311906.702474 read 0x6c280e thru 0x6c3cec (0x14df bytes) 000221[0] 1761311906.750409 read 0x4cc5e2 thru 0x4cd43a (0xe59 bytes) 000222[1] 1761311906.782826 read 0x1a7354 thru 0x1a8d5f (0x1a0c bytes) 000224[1] 1761311906.844250 write 0x1e7482 thru 0x1e818b (0xd0a bytes) 000225[0] 1761311906.844959 trunc from 0x751ef6 to 0x4aa640 (0x2a78b5 bytes) 000227[1] 1761311906.929666 read 0x080f49 thru 0x081950 (0xa08 bytes) 000228[1] 1761311906.956125 mapwrite 0x441fe3 thru 0x442fe3 (0x1001 bytes) 000231[1] 1761311907.078285 mapread 0x2c6a19 thru 0x2c76f8 (0xce0 bytes) 000232[0] 1761311907.126229 trunc from 0x4aa641 to 0x125e98 (0x3847a8 bytes) 000233[1] 1761311907.198650 write 0x9303b7 thru 0x9313b8 (0x1002 bytes) 000234[0] 1761311907.232782 mapread 0x11415c thru 0x11536c (0x1211 bytes) 000235[1] 1761311907.379824 trunc from 0x9313b9 to 0x06c7a7 (0x8c4c11 bytes) 000238[1] 1761311907.493524 trunc from 0x06c7a8 to 0x2ecb6d (0x2803c6 bytes) 000239[0] 1761311907.516419 mapwrite 0x8c001f thru 0x8c066c (0x64e bytes) 000240[1] 1761311907.603470 mapwrite 0x795e39 thru 0x796f68 (0x1130 bytes) 000242[1] 1761311907.701769 trunc from 0x8c066d to 0x2cb5a0 (0x5f50cc bytes) 000244[1] 1761311907.778165 read 0x02a153 thru 0x02b5bc (0x146a bytes) 000245[0] 1761311907.821118 read 0x103481 thru 0x10432e (0xeae bytes) 000246[1] 1761311907.871020 write 0x9406bd thru 0x941d5c (0x16a0 bytes) 000247[1] 1761311907.883340 trunc from 0x941d5d to 0x37eb69 (0x5c31f3 bytes) 000248[1] 1761311907.902250 read 0x21730e thru 0x217b61 (0x854 bytes) 000249[1] 1761311907.948121 write 0x48efcc thru 0x49017b (0x11b0 bytes) 000250[1] 1761311907.950107 read 0x219ab3 thru 0x21ac0a (0x1158 bytes) 000251[0] 1761311907.974844 mapread 0x434b8a thru 0x435a2b (0xea2 bytes) 000252[1] 1761311908.074781 write 0x8595bd thru 0x859c07 (0x64b bytes) 000255[0] 1761311908.169311 read 0x307da1 thru 0x309c03 (0x1e63 bytes) 000256[0] 1761311908.240543 write 0x80707f thru 0x80815d (0x10df bytes) 000257[1] 1761311908.373060 mapwrite 0x2e83e1 thru 0x2e8a6c (0x68c bytes) 000258[0] 1761311908.561838 trunc from 0x859c08 to 0x3da300 (0x47f907 bytes) 000259[1] 1761311908.616940 read 0x25aeca thru 0x25b9c6 (0xafd bytes) 000261[1] 1761311908.673346 read 0x160fb9 thru 0x161972 (0x9ba bytes) 000262[1] 1761311908.724595 trunc from 0x3da301 to 0x5156ab (0x13b3ab bytes) 000263[1] 1761311908.818613 mapwrite 0x647ca8 thru 0x649013 (0x136c bytes) 000265[0] 1761311909.029439 mapwrite 0x42c627 thru 0x42e491 (0x1e6b bytes) 000266[1] 1761311909.128709 read 0x323a59 thru 0x324404 (0x9ac bytes) 000267[0] 1761311909.203858 trunc from 0x649014 to 0x28c8cd (0x3bc746 bytes) 000270[0] 1761311909.346056 write 0x1717b6 thru 0x172d4e (0x1599 bytes) 000274[1] 1761311909.392978 mapread 0x02fcd9 thru 0x0310a5 (0x13cd bytes) 000276[0] 1761311909.465862 trunc from 0x28c8ce to 0x55af9f (0x2ce6d2 bytes) 000278[0] 1761311909.553704 mapread 0x4ac30b thru 0x4aca78 (0x76e bytes) 000279[1] 1761311909.558982 write 0x09a2b2 thru 0x09bd70 (0x1abf bytes) 000280[1] 1761311909.620187 trunc from 0x55afa0 to 0x020dc0 (0x53a1df bytes) 000282[1] 1761311909.686987 write 0x6ccd1f thru 0x6ccede (0x1c0 bytes) 000287[1] 1761311909.751520 mapwrite 0x06ace5 thru 0x06c5b6 (0x18d2 bytes) 000289[0] 1761311909.859124 read 0x144994 thru 0x145e7e (0x14eb bytes) 000290[1] 1761311909.895833 trunc from 0x6ccedf to 0x95fb71 (0x292c93 bytes) 000292[1] 1761311909.931790 write 0x3a788c thru 0x3a7958 (0xcd bytes) 000293[0] 1761311909.971973 mapwrite 0x3453da thru 0x346b8e (0x17b5 bytes) 000294[1] 1761311910.148297 mapread 0x522aee thru 0x523901 (0xe14 bytes) 000298[0] 1761311910.285080 trunc from 0x95fb72 to 0x67ebdd (0x2e0f94 bytes) 000299[1] 1761311910.353408 read 0x229f3f thru 0x22a709 (0x7cb bytes) 000300[0] 1761311910.438294 trunc from 0x67ebde to 0x6fec86 (0x800a9 bytes) 000301[1] 1761311910.480277 mapread 0x34ca40 thru 0x34de0b (0x13cc bytes) 000302[1] 1761311910.591911 mapread 0x10b883 thru 0x10c940 (0x10be bytes) 000305[0] 1761311910.668780 mapwrite 0x007e30 thru 0x008d19 (0xeea bytes) 000306[0] 1761311910.834871 write 0x1a4edc thru 0x1a6025 (0x114a bytes) 000307[1] 1761311910.904007 write 0x543bfb thru 0x5454c5 (0x18cb bytes) 000308[0] 1761311910.953009 mapwrite 0x35166d thru 0x352b88 (0x151c bytes) 000309[0] 1761311911.083665 mapwrite 0x518b82 thru 0x51a3c8 (0x1847 bytes) 000310[1] 1761311911.168605 trunc from 0x6fec87 to 0x656db7 (0xa7ecf bytes) 000312[0] 1761311911.300389 mapwrite 0x986b17 thru 0x988942 (0x1e2c bytes) 000316[1] 1761311911.650661 read 0x4b2f70 thru 0x4b44ed (0x157e bytes) 000317[1] 1761311911.695082 mapwrite 0x4cb398 thru 0x4cd1b7 (0x1e20 bytes) 000318[1] 1761311911.846653 write 0x31bc3a thru 0x31be32 (0x1f9 bytes) 000319[1] 1761311911.885105 mapread 0x364258 thru 0x36504c (0xdf5 bytes) 000321[1] 1761311911.924380 mapwrite 0x2ff2ee thru 0x300d9a (0x1aad bytes) 000322[1] 1761311912.015029 trunc from 0x988943 to 0x517588 (0x4713ba bytes) 000326[0] 1761311912.074266 write 0x98221b thru 0x9841b7 (0x1f9d bytes) 000327[0] 1761311912.104616 read 0x3a299f thru 0x3a2fe2 (0x644 bytes) 000328[1] 1761311912.129996 mapread 0x20c1eb thru 0x20cfc7 (0xddd bytes) 000329[1] 1761311912.153538 mapread 0x5dbc60 thru 0x5dbffe (0x39f bytes) 000330[0] 1761311912.195928 write 0x6f7083 thru 0x6f81c8 (0x1146 bytes) 000333[0] 1761311912.350925 mapread 0x64af91 thru 0x64cc2a (0x1c9a bytes) 000334[1] 1761311912.352031 read 0x50491f thru 0x506427 (0x1b09 bytes) 000338[0] 1761311912.503901 mapwrite 0x4ed812 thru 0x4ee51c (0xd0b bytes) 000339[0] 1761311912.547666 trunc from 0x9841b8 to 0x2d2072 (0x6b2145 bytes) 000341[1] 1761311912.656854 write 0x16e08b thru 0x16fd03 (0x1c79 bytes) 000342[0] 1761311912.691265 write 0x3d4262 thru 0x3d46b4 (0x453 bytes) 000344[0] 1761311912.737640 read 0x34c74f thru 0x34c764 (0x16 bytes) 000345[0] 1761311912.785542 write 0x59b125 thru 0x59bf07 (0xde3 bytes) 000346[0] 1761311912.812186 read 0x41ced0 thru 0x41e050 (0x1181 bytes) 000347[0] 1761311912.862635 mapwrite 0x6fb726 thru 0x6fc3a5 (0xc80 bytes) 000348[0] 1761311912.974388 trunc from 0x6fc3a6 to 0x5cc663 (0x12fd42 bytes) 000350[1] 1761311913.024380 read 0x204291 thru 0x205f33 (0x1ca3 bytes) 000351[0] 1761311913.082741 mapwrite 0x3942c9 thru 0x394382 (0xba bytes) 000352[0] 1761311913.226851 trunc from 0x5cc664 to 0x5ef37a (0x22d17 bytes) 000353[0] 1761311913.267624 trunc from 0x5ef37b to 0x2bdac2 (0x3318b8 bytes) 000354[0] 1761311913.307524 mapwrite 0x61149b thru 0x61213f (0xca5 bytes) 000355[0] 1761311913.468137 write 0x7ae3f5 thru 0x7aef5e (0xb6a bytes) 000357[0] 1761311913.484839 mapwrite 0x216f2c thru 0x218e23 (0x1ef8 bytes) 000358[1] 1761311913.624264 mapwrite 0x86839e thru 0x869bf7 (0x185a bytes) 000359[1] 1761311913.915106 write 0x84196f thru 0x843040 (0x16d2 bytes) 000360[0] 1761311913.944140 write 0x6654ec thru 0x6664b0 (0xfc5 bytes) 000361[0] 1761311914.008126 mapread 0x16993c thru 0x169b02 (0x1c7 bytes) 000362[0] 1761311914.046096 mapread 0x5d669b thru 0x5d7675 (0xfdb bytes) 000368[1] 1761311914.095754 trunc from 0x869bf8 to 0x4d9deb (0x38fe0c bytes) 000369[0] 1761311914.200643 write 0x11055e thru 0x11076a (0x20d bytes) 000370[0] 1761311914.261868 read 0x307461 thru 0x3093c2 (0x1f62 bytes) 000371[1] 1761311914.325703 read 0x2c4ffd thru 0x2c5d87 (0xd8b bytes) 000373[1] 1761311914.392660 trunc from 0x4d9dec to 0x1f1f0a (0x2e7ee1 bytes) 000375[1] 1761311914.490285 read 0x0050ef thru 0x006110 (0x1022 bytes) 000376[1] 1761311914.645398 mapwrite 0x742bf5 thru 0x743fda (0x13e6 bytes) 000377[1] 1761311914.868369 mapwrite 0x3959b6 thru 0x3964c2 (0xb0d bytes) 000378[1] 1761311914.927298 write 0x5266e0 thru 0x52748e (0xdaf bytes) 000384[0] 1761311915.012947 trunc from 0x743fdb to 0x1566db (0x5ed8ff bytes) 000385[0] 1761311915.090639 write 0x9b447e thru 0x9b4f66 (0xae9 bytes) 000386[0] 1761311915.097662 write 0x0812ca thru 0x0820f7 (0xe2e bytes) 000391[0] 1761311915.231549 mapwrite 0x3c127c thru 0x3c150c (0x291 bytes) 000392[1] 1761311915.291242 mapwrite 0x0b553b thru 0x0b6883 (0x1349 bytes) 000394[1] 1761311915.443872 trunc from 0x9b4f67 to 0x825a31 (0x18f535 bytes) 000399[0] 1761311915.673015 trunc from 0x825a32 to 0x30f156 (0x5168db bytes) 000400[1] 1761311915.798935 write 0x7eafb9 thru 0x7eceb4 (0x1efc bytes) 000402[0] 1761311915.876601 mapread 0x3dd6bd thru 0x3df53c (0x1e80 bytes) 000403[0] 1761311916.021497 mapread 0x0deb20 thru 0x0df2cc (0x7ad bytes) 000404[0] 1761311916.078424 mapread 0x0424b2 thru 0x0442db (0x1e2a bytes) 000405[0] 1761311916.118300 trunc from 0x7eceb5 to 0x54e8dc (0x29e5d8 bytes) 000407[0] 1761311916.284620 mapread 0x0282ae thru 0x02922d (0xf80 bytes) 000408[0] 1761311916.366725 mapread 0x25a746 thru 0x25b8da (0x1195 bytes) 000410[0] 1761311916.467822 trunc from 0x54e8dd to 0x41bd16 (0x132bc6 bytes) 000411[1] 1761311916.554939 write 0x20dcb9 thru 0x20e5c7 (0x90f bytes) 000413[0] 1761311916.629298 write 0x0e570d thru 0x0e5891 (0x185 bytes) 000414[1] 1761311916.658319 read 0x01692e thru 0x017a2a (0x10fd bytes) 000415[1] 1761311916.743048 write 0x033eaf thru 0x0344dd (0x62f bytes) 000416[0] 1761311916.812885 write 0x300a28 thru 0x301476 (0xa4f bytes) 000417[0] 1761311916.884378 write 0x12b303 thru 0x12c132 (0xe30 bytes) 000418[1] 1761311916.958003 mapwrite 0x3de40f thru 0x3dec78 (0x86a bytes) 000419[1] 1761311917.249341 write 0x4303da thru 0x430c95 (0x8bc bytes) 000420[0] 1761311917.313725 read 0x419b82 thru 0x41ae11 (0x1290 bytes) 000421[0] 1761311917.434728 mapread 0x3cd9e6 thru 0x3cf3e5 (0x1a00 bytes) 000425[0] 1761311917.533036 write 0x752e3f thru 0x7530c9 (0x28b bytes) 000428[1] 1761311917.565281 mapread 0x316e06 thru 0x31896d (0x1b68 bytes) 000429[0] 1761311917.824322 mapread 0x304d6a thru 0x30543d (0x6d4 bytes) 000430[1] 1761311917.922920 mapwrite 0x477912 thru 0x47853a (0xc29 bytes) 000433[0] 1761311918.125727 read 0x4129a5 thru 0x4137fa (0xe56 bytes) 000434[0] 1761311918.237119 mapwrite 0x167b55 thru 0x16816e (0x61a bytes) 000435[0] 1761311918.508564 read 0x244967 thru 0x245462 (0xafc bytes) 000436[1] 1761311918.619922 read 0x538afa thru 0x539bdf (0x10e6 bytes) 000438[1] 1761311918.766034 write 0x065ea4 thru 0x067a28 (0x1b85 bytes) 000440[1] 1761311918.933331 write 0x1bc6ee thru 0x1bdb06 (0x1419 bytes) 000442[0] 1761311919.042574 trunc from 0x7530ca to 0x8c4c2b (0x171b62 bytes) 000443[1] 1761311919.173699 mapread 0x436449 thru 0x437141 (0xcf9 bytes) 000444[1] 1761311919.227312 mapread 0x8c0a8a thru 0x8c12bb (0x832 bytes) 000445[1] 1761311919.251230 mapread 0x155138 thru 0x155554 (0x41d bytes) 000446[0] 1761311919.344069 mapread 0x7ab57c thru 0x7acc77 (0x16fc bytes) 000448[0] 1761311919.430595 trunc from 0x8c4c2c to 0x03a89d (0x88a38e bytes) 000450[1] 1761311919.536670 mapread 0x0039e4 thru 0x004ebb (0x14d8 bytes) 000451[0] 1761311919.598038 mapwrite 0x646e4f thru 0x6471ff (0x3b1 bytes) 000455[0] 1761311919.874460 read 0x31ff77 thru 0x320040 (0xca bytes) 000456[1] 1761311919.964301 mapwrite 0x2dbffa thru 0x2ddb6c (0x1b73 bytes) 000458[1] 1761311920.222035 write 0x4d80a4 thru 0x4d960d (0x156a bytes) 000459[1] 1761311920.301310 mapwrite 0x6002a5 thru 0x600475 (0x1d1 bytes) 000460[0] 1761311920.465548 mapwrite 0x5b7651 thru 0x5b7bfc (0x5ac bytes) 000462[1] 1761311920.541741 read 0x47ffe4 thru 0x4800b2 (0xcf bytes) 000463[1] 1761311920.575721 trunc from 0x647200 to 0x8ea7ed (0x2a35ee bytes) 000464[1] 1761311920.630732 mapread 0x701ce6 thru 0x701e7f (0x19a bytes) 000465[0] 1761311920.783969 mapwrite 0x69e96c thru 0x6a0106 (0x179b bytes) 000466[0] 1761311921.018597 read 0x7dc783 thru 0x7dd5fe (0xe7c bytes) 000468[1] 1761311921.178839 mapwrite 0x03f399 thru 0x040ae1 (0x1749 bytes) 000469[0] 1761311921.320095 mapwrite 0x5fa0bb thru 0x5fb5ee (0x1534 bytes) 000472[1] 1761311921.673829 read 0x316074 thru 0x316b9e (0xb2b bytes) 000476[1] 1761311921.934244 mapread 0x4c5b80 thru 0x4c5cec (0x16d bytes) 000477[0] 1761311921.934860 read 0x2505ca thru 0x251591 (0xfc8 bytes) 000479[1] 1761311922.214677 mapwrite 0x6e7d78 thru 0x6e8c18 (0xea1 bytes) 000481[0] 1761311922.391712 read 0x6d237a thru 0x6d384e (0x14d5 bytes) 000482[0] 1761311922.489686 trunc from 0x8ea7ee to 0x66205a (0x288793 bytes) 000487[0] 1761311922.715481 mapwrite 0x839205 thru 0x839e3e (0xc3a bytes) 000488[1] 1761311922.865739 write 0x66199d thru 0x6635fa (0x1c5e bytes) 000490[1] 1761311922.974761 mapwrite 0x7e2b63 thru 0x7e423a (0x16d8 bytes) 000491[0] 1761311923.110534 mapread 0x452386 thru 0x453548 (0x11c3 bytes) 000493[0] 1761311923.221077 trunc from 0x839e3f to 0x04fe5b (0x7e9fe3 bytes) 000494[0] 1761311923.335366 read 0x004807 thru 0x004f79 (0x773 bytes) 000499[0] 1761311923.551089 mapread 0x02de64 thru 0x02eab0 (0xc4d bytes) All operations completed A-OK! PASS 16b (34s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 09:18:49 (1761311929) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (4s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 09:18:54 (1761311934) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.352929 s, 5.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.184865 s, 11.3 MB/s ldlm.namespaces.MGC192.168.204.105@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9b4c870cd000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9b4c99097800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9b4c99097800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9b4c99097800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.05551 s, 11.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.45786 s, 42.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.30317 s, 45.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.61955 s, 18.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.23278 s, 14.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.6726 s, 39.2 MB/s PASS 16d (41s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 09:19:35 (1761311975) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.194145 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878872 s, 11.9 MB/s PASS 16e (7s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 09:19:41 (1761311981) TIMEOUT PASS 16f (27s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 09:20:08 (1761312008) TIMEOUT PASS 16g (28s) == sanityn test 16i: read after truncate file ============ 09:20:36 (1761312036) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (7s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 09:20:43 (1761312043) fail_loc=0x8000030a PASS 17 (8s) == sanityn test 18: mmap sanity check =========================================================================================== 09:20:52 (1761312052) mmap test1: basic mmap operation (PASS, 0.11632s) mmap test2: MAP_PRIVATE not write back (PASS, 0.055258s) mmap test3: concurrent mmap ops on two nodes (PASS, 22.671s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.3486s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.1879s) mmap test6: check mmap write/read content on two nodes (PASS, 0.19227s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0689s) PASS 18 (34s) == sanityn test 19: test concurrent uncached read races ========================================================================= 09:21:25 (1761312085) oleg405-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg405-client: oleg405-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (3s) == sanityn test 20: test extra readahead page left in cache ============================================================== 09:21:28 (1761312088) PASS 20 (5s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 09:21:33 (1761312093) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (5s) == sanityn test 23: others should see updated atime while another read============================================================== 09:21:38 (1761312098) now is 1761312100 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 new atime is 1761312161 PASS 23 (68s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 09:22:46 (1761312166) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 374 115.0K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 374 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 4.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118102 374 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 374 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118102 374 117728 1% /mnt/lustre[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118102 374 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (6s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 09:22:52 (1761312172) PASS 24b (5s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 09:22:57 (1761312177) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (6s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 09:23:03 (1761312183) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (3s) == sanityn test 26a: allow mtime to get older ============ 09:23:06 (1761312186) utime: good mknod times 1761312186 <= 1761312187 <= 1761312187 for /mnt/lustre/f26a utime: good mknod times 1761312186 <= 1761312187 <= 1761312187 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (7s) == sanityn test 26b: sync mtime between ost and mds ====== 09:23:13 (1761312193) PASS 26b (9s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 09:23:26 (1761312206) 0 PASS 30 (8s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 09:23:34 (1761312214) fail_loc=0x314 PASS 31a (7s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 09:23:41 (1761312221) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg405-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.105@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (20s) == sanityn test 31r: open-rename(replace) race =========== 09:24:01 (1761312241) -rw-r--r-- 1 root root 0 Oct 24 09:24 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (9s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 09:24:11 (1761312251) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (3s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 09:24:15 (1761312255) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 09:24:17 (1761312257) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (3s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 09:24:20 (1761312260) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (3s) == sanityn test 34: no lock timeout under IO ============= 09:24:23 (1761312263) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b4c870cd000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9b4c99097800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff9b4c870cd000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff9b4c99097800.ost_server_uuid in FULL state after 0 sec oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9b4c870cd000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9b4c99097800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff9b4c870cd000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff9b4c99097800.ost_server_uuid in IDLE state after 0 sec oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b4c870cd000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9b4c99097800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff9b4c870cd000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9b4c99097800.ost_server_uuid in FULL state after 0 sec oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9b4c870cd000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9b4c99097800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff9b4c870cd000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff9b4c99097800.ost_server_uuid in IDLE state after 0 sec oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b4c870cd000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9b4c99097800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff9b4c870cd000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9b4c99097800.ost_server_uuid in FULL state after 0 sec oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9b4c870cd000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9b4c99097800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff9b4c870cd000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff9b4c99097800.ost_server_uuid in IDLE state after 0 sec PASS 34 (119s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 09:26:22 (1761312382) fail_loc=0x80000317 Race attempt 0 Wait for 44690 44767 for 60 sec... - open/close 895 (time 1761312398.05 total 10.00 last 89.46) - open/close 1709 (time 1761312408.05 total 20.01 last 81.38) - open/close 2602 (time 1761312418.05 total 30.01 last 89.29) - open/close 3394 (time 1761312428.06 total 40.02 last 79.12) total: 4000 open/close in 47.28 seconds: 84.61 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.105@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.204.105@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (70s) == sanityn test 36: handle ESTALE/open-unlink correctly == 09:27:32 (1761312452) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg405-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.682174 s, 76.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.02861 s, 51.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7471104) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.0496 s, 50.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.949976 s, 55.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7522304) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.880738 s, 59.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7467008) after(7518208) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.941418 s, 55.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7518208) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.866287 s, 60.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7522304) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.0856 s, 48.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.98156 s, 53.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.697917 s, 75.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7522304) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.914254 s, 57.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7524352) after_dd(7469056) after(7520256) PASS 36 (274s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 09:32:06 (1761312726) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 - create 1791 (time 1761312739.50 total 10.00 last 179.06) - create 3545 (time 1761312749.50 total 20.00 last 175.36) - create 5385 (time 1761312759.51 total 30.01 last 183.91) - create 7008 (time 1761312769.51 total 40.01 last 162.28) - create 8644 (time 1761312779.52 total 50.02 last 163.45) total: 10000 create in 57.08 seconds: 175.20 ops/second PASS 37 (65s) == sanityn test 39a: test from 11063 ============================================================================================ 09:33:11 (1761312791) repeat after cancel_lru_locks PASS 39a (5s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 09:33:16 (1761312796) repeat after cancel_lru_locks PASS 39b (6s) == sanityn test 39c: check truncate mtime update ================================================================================ 09:33:22 (1761312802) repeat after cancel_lru_locks PASS 39c (6s) == sanityn test 39d: sync write should update mtime ====== 09:33:28 (1761312808) fail_loc=0x411 fail_loc=0 PASS 39d (5s) == sanityn test 40a: pdirops: create vs others ======================================================================== 09:33:33 (1761312813) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (12s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 09:33:45 (1761312825) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (13s) == sanityn test 40c: pdirops: link and others ======================================================================== 09:33:58 (1761312838) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (13s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 09:34:11 (1761312851) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (14s) == sanityn test 40e: pdirops: rename and others ======================================================================== 09:34:25 (1761312865) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (13s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 09:34:38 (1761312878) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (11s) == sanityn test 41b: pdirops: create vs create ======================================================================== 09:34:49 (1761312889) open: File exists Conflict PASS 41b (10s) == sanityn test 41c: pdirops: create vs link ======================================================================== 09:34:59 (1761312899) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (9s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 09:35:08 (1761312908) Conflict PASS 41d (12s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 09:35:20 (1761312920) Conflict PASS 41e (12s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 09:35:32 (1761312932) Conflict PASS 41f (10s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 09:35:42 (1761312942) Conflict PASS 41g (10s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 09:35:52 (1761312952) Conflict PASS 41h (10s) == sanityn test 41i: reint_open: create vs create ======== 09:36:02 (1761312962) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (957s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 09:51:59 (1761313919) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (10s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 09:52:09 (1761313929) open: File exists Conflict PASS 42b (10s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 09:52:20 (1761313940) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (12s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 09:52:31 (1761313951) Conflict PASS 42d (13s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 09:52:44 (1761313964) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (12s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 09:52:56 (1761313976) Conflict PASS 42f (11s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 09:53:07 (1761313987) Conflict PASS 42g (11s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 09:53:18 (1761313998) Conflict PASS 42h (11s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 09:53:29 (1761314009) PASS 43a (39s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 09:54:08 (1761314048) Conflict PASS 43b (9s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 09:54:18 (1761314058) Conflict PASS 43c (9s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 09:54:26 (1761314066) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (10s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 09:54:36 (1761314076) Conflict PASS 43e (10s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 09:54:46 (1761314086) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (10s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 09:54:56 (1761314096) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (9s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 09:55:05 (1761314105) Conflict PASS 43h (10s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 09:55:15 (1761314115) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (3s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 09:55:18 (1761314118) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (101s) == sanityn test 43k: unlink vs create ==================== 09:56:59 (1761314219) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (911s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 10:12:10 (1761315130) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Oct 24 10:12:18 EDT 2025 PASS 44a (12s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 10:12:23 (1761315143) open: File exists Conflict PASS 44b (14s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 10:12:37 (1761315157) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (11s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 10:12:48 (1761315168) Conflict PASS 44d (10s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 10:12:59 (1761315179) Conflict PASS 44e (11s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 10:13:09 (1761315189) Conflict PASS 44f (11s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 10:13:20 (1761315200) Conflict PASS 44g (10s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 10:13:30 (1761315210) Conflict PASS 44h (11s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 10:13:41 (1761315221) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (3s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 10:13:44 (1761315224) PASS 45a (87s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 10:15:11 (1761315311) Conflict PASS 45b (10s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 10:15:21 (1761315321) Conflict PASS 45c (11s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 10:15:32 (1761315332) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (11s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 10:15:44 (1761315344) Conflict PASS 45e (11s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 10:15:55 (1761315355) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (11s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 10:16:06 (1761315366) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (10s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 10:16:16 (1761315376) Conflict PASS 45h (11s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 10:16:27 (1761315387) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (3s) == sanityn test 45j: read vs rename ====================== 10:16:30 (1761315390) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (1058s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 10:34:08 (1761316448) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (12s) == sanityn test 46b: pdirops: link vs create ======================================================================== 10:34:20 (1761316460) open: File exists Conflict PASS 46b (12s) == sanityn test 46c: pdirops: link vs link ======================================================================== 10:34:32 (1761316472) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (10s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 10:34:43 (1761316483) Conflict PASS 46d (14s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 10:34:56 (1761316496) Conflict PASS 46e (13s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 10:35:10 (1761316510) Conflict PASS 46f (13s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 10:35:22 (1761316522) Conflict PASS 46g (11s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 10:35:34 (1761316534) Conflict PASS 46h (12s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 10:35:45 (1761316545) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (3s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 10:35:48 (1761316548) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (3s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 10:35:51 (1761316551) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (3s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 10:35:54 (1761316554) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (4s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 10:35:58 (1761316558) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (3s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 10:36:01 (1761316561) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (3s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 10:36:04 (1761316564) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (3s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 10:36:07 (1761316567) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (3s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 10:36:10 (1761316570) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.138551 s, 73.9 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (12s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 10:36:22 (1761316582) 0+1 records in 0+1 records out 158 bytes copied, 0.033227 s, 4.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242331) - No such process PASS 51a (8s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 10:36:30 (1761316590) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0125005 s, 81.9 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647044 s, 16.2 MB/s 3145728 PASS 51b (18s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 10:36:48 (1761316608) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0175166 s, 58.5 kB/s PASS 51c (10s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 10:36:58 (1761316618) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459071 s, 22.8 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (7s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 10:37:05 (1761316625) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2430 0x97e 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2430 0x97e 0 read lease(1) has applied. PASS 51e (8s) == sanityn test 54: rename locking ======================= 10:37:13 (1761316633) llite.lustre-ffff9b4c870cd000.xattr_cache=1 llite.lustre-ffff9b4c99097800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:37:14.000000000 -0400 Modify: 2025-10-24 10:37:14.000000000 -0400 Change: 2025-10-24 10:37:14.000000000 -0400 Birth: 2025-10-24 10:37:14.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:37:20.000000000 -0400 Modify: 2025-10-24 10:37:21.000000000 -0400 Change: 2025-10-24 10:37:21.000000000 -0400 Birth: 2025-10-24 10:37:21.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:37:27.000000000 -0400 Modify: 2025-10-24 10:37:27.000000000 -0400 Change: 2025-10-24 10:37:27.000000000 -0400 Birth: 2025-10-24 10:37:27.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:37:27.000000000 -0400 Modify: 2025-10-24 10:37:28.000000000 -0400 Change: 2025-10-24 10:37:28.000000000 -0400 Birth: 2025-10-24 10:37:27.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:37:33.000000000 -0400 Modify: 2025-10-24 10:37:34.000000000 -0400 Change: 2025-10-24 10:37:34.000000000 -0400 Birth: 2025-10-24 10:37:33.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:37:33.000000000 -0400 Modify: 2025-10-24 10:37:34.000000000 -0400 Change: 2025-10-24 10:37:34.000000000 -0400 Birth: 2025-10-24 10:37:33.000000000 -0400 PASS 54 (30s) == sanityn test 55a: rename vs unlink target dir ========= 10:37:44 (1761316664) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (14s) == sanityn test 55b: rename vs unlink source dir ========= 10:37:57 (1761316677) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (16s) == sanityn test 55c: rename vs unlink orphan target dir == 10:38:13 (1761316693) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (18s) == sanityn test 55d: rename file vs link ================= 10:38:31 (1761316711) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (13s) == sanityn test 60: Verify data_version behaviour ======== 10:38:44 (1761316724) PASS 60 (5s) == sanityn test 70a: cd directory && rm directory ======== 10:38:49 (1761316729) PASS 70a (5s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 10:38:55 (1761316735) PASS 70b (5s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 10:39:00 (1761316740) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0237292 s, 2.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 10:39:03 (1761316743) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.012387 s, 5.3 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (3s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 10:39:06 (1761316746) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 10:39:09 (1761316749) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0236765 s, 2.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (4s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 10:39:13 (1761316753) llite.lustre-ffff9b4c870cd000.xattr_cache=1 llite.lustre-ffff9b4c99097800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (6s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 10:39:19 (1761316759) llite.lustre-ffff9b4c870cd000.xattr_cache=1 llite.lustre-ffff9b4c99097800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (7s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 10:39:26 (1761316766) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0211395 s, 48.4 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0353821 s, 28.9 kB/s 252388: get lock2 252390: get lock1 252388: done 252388: get lock1 252390: done 252390: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252390: cannot get lock2: Resource deadlock avoided 252390: exit rc=0 252388: done 252388: exit rc=0 PASS 74 (9s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 10:39:35 (1761316775) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0779958 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330665 s, 31.7 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 252965 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (12s) == sanityn test 76: Verify MDT open_files listing ======== 10:39:47 (1761316787) opening files: ........ closing files: ........ PASS 76 (53s) == sanityn test 77a: check FIFO NRS policy =============== 10:40:41 (1761316841) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.431239 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431982 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419356 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643749 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125734 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537021 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644169 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100638 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675433 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784118 s, 13.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221389 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206137 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200867 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223881 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257818 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308149 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347701 s, 3.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552585 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601095 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678412 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0628203 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678583 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693465 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0749256 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102107 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071594 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100792 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803387 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078474 s, 13.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0827869 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764449 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797252 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762025 s, 13.8 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (7s) == sanityn test 77b: check CRR-N NRS policy ============== 10:40:48 (1761316848) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.333272 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261101 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617951 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712318 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107155 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150278 s, 7.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179293 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172473 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201646 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19325 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230737 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187687 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194867 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235703 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228222 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191486 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261875 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708384 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718327 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800194 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875662 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537191 s, 19.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642951 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941421 s, 11.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815323 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657764 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744008 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10513 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732605 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747584 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811344 s, 12.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813899 s, 12.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0827253 s, 12.7 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.301792 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398439 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835888 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0950758 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931159 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734533 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899525 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102865 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857831 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503768 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660309 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195766 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232752 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228211 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22301 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200734 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257596 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00293385 s, 357 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604611 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673505 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593882 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570359 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014473 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853197 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00312477 s, 336 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803196 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931065 s, 11.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388302 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797584 s, 13.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057809 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435406 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161389 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00423319 s, 248 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (11s) == sanityn test 77c: check ORR NRS policy ================ 10:40:59 (1761316859) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.296812 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397812 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477698 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966233 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108641 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139955 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133851 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182844 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169343 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229854 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209641 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213334 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229837 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253528 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250828 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267644 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310772 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937172 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736077 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00369349 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871414 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00337828 s, 310 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394764 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00371688 s, 282 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00259014 s, 405 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111139 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0555605 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115504 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112364 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116673 s, 9.0 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821891 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655895 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0714825 s, 14.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.425827 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0598004 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868272 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135565 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115143 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165361 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197806 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176696 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286249 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177235 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218298 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235875 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253008 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399718 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317811 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297534 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.451304 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00235284 s, 446 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00431501 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00242632 s, 432 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110763 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172578 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00464198 s, 226 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224189 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17034 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816942 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067341 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683913 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033572 s, 31.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537585 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283074 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306992 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307142 s, 34.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.324502 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107225 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954964 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114727 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946373 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123265 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192957 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963356 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135457 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138331 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297494 s, 3.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271885 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322268 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27694 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340498 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.345586 s, 3.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409255 s, 2.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00208059 s, 504 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041749 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799668 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282413 s, 37.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070426 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00234635 s, 447 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00193859 s, 541 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355526 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169514 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00240361 s, 436 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00229869 s, 456 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101458 s, 10.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026922 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158893 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364653 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664492 s, 158 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (16s) == sanityn test 77d: check TRR nrs policy ================ 10:41:16 (1761316876) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.573348 s, 29.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603318 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110068 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446944 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639528 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168075 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195668 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130928 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105032 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843841 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269845 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110491 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260975 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271679 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179298 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.388862 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26513 s, 4.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128574 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128768 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773397 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829708 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00238007 s, 441 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104649 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212291 s, 4.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960501 s, 10.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142922 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584197 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586137 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106284 s, 9.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126087 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226553 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0896688 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05431 s, 19.3 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.510908 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0714415 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478266 s, 21.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360725 s, 29.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134784 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0542052 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911936 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0600715 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724771 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134132 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156513 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161768 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23452 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170655 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254202 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269686 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317162 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207437 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031178 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500694 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198759 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845737 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280972 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320209 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883097 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971056 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966789 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590767 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690051 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642684 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566946 s, 18.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694734 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747532 s, 14.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=4 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.410329 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0633938 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082755 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670171 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107001 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154937 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209172 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216817 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230036 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162023 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224974 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229375 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23774 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258809 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196637 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239168 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303584 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723596 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765582 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924742 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957782 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870683 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868438 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836393 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101899 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860262 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139807 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095711 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10466 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180044 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084186 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865265 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723291 s, 14.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (14s) == sanityn test 77e: check TBF NID nrs policy ============ 10:41:30 (1761316890) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.5@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.273768 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033908 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764859 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726639 s, 14.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109552 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702501 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0418559 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140882 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980588 s, 10.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114468 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1561 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132783 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122865 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134654 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101703 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147842 s, 7.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152266 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14361 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123649 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139662 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147712 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15767 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161032 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158025 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175233 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165831 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150554 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157335 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159642 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181633 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159848 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164275 s, 6.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175991 s, 6.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.330704 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0437031 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770318 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151204 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11846 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142501 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17617 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186133 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18009 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165015 s, 6.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180534 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200332 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222771 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206248 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183703 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21922 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257612 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792895 s, 13.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115514 s, 90.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837005 s, 125 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771181 s, 13.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571946 s, 18.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584527 s, 17.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603363 s, 17.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0646111 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677167 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00227422 s, 461 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00238125 s, 440 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801365 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624893 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096882 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159175 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208172 s, 50.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.303738 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574212 s, 18.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047951 s, 21.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816837 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860358 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134434 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122007 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139175 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092484 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831088 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179388 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184216 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263275 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122102 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193072 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283569 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337738 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432533 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409273 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664687 s, 15.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381438 s, 27.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455247 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471936 s, 22.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360835 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511604 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0443396 s, 23.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078784 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472115 s, 22.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057941 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468042 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594134 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651827 s, 16.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.341057 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532776 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062731 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860236 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108622 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159436 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15138 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157664 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20328 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179762 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205391 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209407 s, 5.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213284 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229796 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247618 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250813 s, 4.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271887 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0545795 s, 19.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655573 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936961 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591902 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738737 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07113 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733458 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068088 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830476 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736533 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663742 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835578 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109663 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131424 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114949 s, 9.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136952 s, 7.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (18s) == sanityn test 77f: check TBF JobID nrs policy ========== 10:41:48 (1761316908) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.317479 s, 52.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441778 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0691762 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654044 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056475 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100728 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137334 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175039 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147291 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151686 s, 6.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149646 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209968 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209616 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264291 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209116 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200885 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34728 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339421 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684068 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381101 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392554 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112139 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346716 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0437376 s, 24.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466334 s, 22.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465425 s, 22.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426625 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05136 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432573 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446382 s, 23.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490624 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507292 s, 20.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558606 s, 18.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.33938 s, 49.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354621 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118893 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590262 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111757 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140731 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849435 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07401 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1295 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190669 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298932 s, 3.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258883 s, 4.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25841 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284404 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308548 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230684 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195941 s, 5.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=7 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085613 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792045 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00240631 s, 436 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00238063 s, 440 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988554 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128093 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901366 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00228308 s, 459 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116829 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.064886 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536088 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135337 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508152 s, 20.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922645 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500336 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456906 s, 22.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.305803 s, 54.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040351 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0602913 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13393 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126098 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151929 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13364 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115606 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175728 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187565 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220226 s, 4.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195946 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231431 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217155 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215432 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269269 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306321 s, 3.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358193 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044067 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409226 s, 25.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324979 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583757 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485593 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541779 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524042 s, 20.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05576 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047515 s, 22.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785791 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526118 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852942 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586556 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592404 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0754661 s, 13.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.43007 s, 39.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624657 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627904 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472564 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147847 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13229 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197981 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20049 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222522 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234918 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202919 s, 5.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9b4c870cd000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201329 s, 5.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236226 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267107 s, 3.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238662 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235302 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313351 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612122 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0589638 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00356925 s, 294 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972737 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0464021 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164203 s, 63.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00529729 s, 198 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364494 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048445 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118898 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0653006 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166296 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00427846 s, 245 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115521 s, 9.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761891 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00493581 s, 212 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (20s) == sanityn test 77g: Change TBF type directly ============ 10:42:08 (1761316928) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (9s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 10:42:17 (1761316937) oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg405-client: oleg405-server: ssh exited with exit code 2 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 12 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 12 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 12 PASS 77h (11s) == sanityn test 77i: Change rank of TBF rule ============= 10:42:28 (1761316948) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (21s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 10:42:50 (1761316970) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.423998 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482338 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683705 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700343 s, 15.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227853 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217089 s, 4.8 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183854 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160303 s, 6.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229232 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250568 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253916 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23912 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284312 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249305 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271696 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25203 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270385 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572108 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466325 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620018 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050649 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113168 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0814325 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531354 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531694 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372466 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110645 s, 9.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572045 s, 18.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575587 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615785 s, 17.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0923041 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485114 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642016 s, 16.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.58531 s, 15.9 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5891 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (62s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 10:43:52 (1761317032) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.785965 s, 21.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10185 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10879 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123465 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177349 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19252 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210551 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177514 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220613 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285945 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304205 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318571 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.370594 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395318 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346194 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3434 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.424685 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00254249 s, 412 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932008 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0037623 s, 279 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.003545 s, 296 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00235755 s, 445 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00525613 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088607 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787909 s, 13.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742119 s, 14.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857202 s, 12.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869816 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120122 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106929 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103836 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594098 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261488 s, 40.1 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9152 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9395 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.787662 s, 21.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.08631 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0543015 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10466 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965075 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894036 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947924 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107065 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121631 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280745 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255914 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.289039 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355891 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358912 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356357 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37673 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34823 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174812 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335557 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184238 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552794 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238625 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070778 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278241 s, 37.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322132 s, 32.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566437 s, 18.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489097 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591449 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388034 s, 27.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736926 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040312 s, 26.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424101 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04227 s, 24.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7419 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9537 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (148s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 10:46:20 (1761317180) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.323884 s, 51.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552727 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074251 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126214 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175722 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169252 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183515 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179946 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164029 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161928 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205086 s, 5.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241717 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262724 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226505 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24718 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260752 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277074 s, 3.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719831 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119158 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00229231 s, 457 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0872235 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916392 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079778 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692651 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11993 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057135 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138451 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831657 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522573 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037149 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325146 s, 32.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986612 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484241 s, 21.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.05212 s, 17.3 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.8747 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.5@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.5@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.398495 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516237 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730649 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547863 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12812 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170746 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190428 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141908 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22479 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166445 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164002 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216076 s, 4.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203846 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191406 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215941 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253187 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223568 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503288 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0570675 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0589587 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120362 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0859348 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138058 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880003 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682412 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532301 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.08548 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630815 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087449 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808241 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083483 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982013 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101682 s, 10.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.63814 s, 15.8 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.89991 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.5@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.338787 s, 49.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700532 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434219 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160138 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130226 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16098 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19051 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23859 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243596 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263138 s, 4.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215928 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24641 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218135 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258731 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25717 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27155 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294646 s, 3.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173074 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00275439 s, 381 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00318181 s, 330 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00365237 s, 287 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00263678 s, 398 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0780389 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107607 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625744 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517099 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00257722 s, 407 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213879 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475603 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454875 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365815 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285338 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354959 s, 29.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.34547 s, 16.5 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.04486 s, 20.8 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.508976 s, 33.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294464 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362234 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.054368 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789264 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877828 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119537 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14412 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11103 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152963 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120765 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152945 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118837 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139844 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177413 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227639 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20098 s, 5.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599295 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00446083 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109708 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386855 s, 27.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307658 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227888 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579204 s, 18.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216994 s, 48.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00408826 s, 256 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416695 s, 252 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547969 s, 19.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202299 s, 51.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417071 s, 25.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00422957 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212843 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115308 s, 90.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.2932 s, 10.2 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.94024 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.5869 s, 12.2 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.97411 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.873408 s, 19.2 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071146 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0962959 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128994 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135259 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171121 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168444 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228525 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241324 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215418 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212432 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232682 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244153 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352423 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359143 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311548 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393766 s, 2.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639585 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965202 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664019 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149234 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0696121 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426986 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.035778 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14204 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119725 s, 8.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099014 s, 106 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340021 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00243863 s, 430 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129095 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00223824 s, 468 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181177 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033879 s, 31.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9168 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5984 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.407586 s, 41.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172122 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0439948 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777324 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039939 s, 26.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804398 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901571 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101481 s, 10.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940206 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372983 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166502 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183987 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229244 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208423 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209151 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25583 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273605 s, 3.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022254 s, 47.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527207 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320047 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349779 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034846 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398225 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0418197 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440255 s, 23.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700131 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466512 s, 22.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532664 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535966 s, 19.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539701 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573444 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541631 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708654 s, 14.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.79109 s, 15.4 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.93431 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.489949 s, 34.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0461409 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905614 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805505 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541193 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117686 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796251 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835182 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110965 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156465 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141088 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235239 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224434 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156504 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993562 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210551 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203661 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653049 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00228445 s, 459 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00229957 s, 456 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00217228 s, 483 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059811 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559638 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115941 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00339566 s, 309 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681265 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00261907 s, 400 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109221 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523437 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847659 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066916 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421115 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593565 s, 17.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.5353 s, 10.0 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.93342 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.4424 s, 10.0 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.93202 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (417s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 10:53:17 (1761317597) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (8s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 10:53:25 (1761317605) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg405-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 4.11352 s, 255 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg405-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0519 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (56s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 10:54:22 (1761317662) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.341642 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471404 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466409 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944042 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184092 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102278 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372677 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520678 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118545 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365634 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815366 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144489 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25162 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248095 s, 4.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238666 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270554 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268155 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634992 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659385 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470031 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106745 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523708 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550536 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444468 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126833 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673261 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737475 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940642 s, 11.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9b4c870cd000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078896 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809752 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105112 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843241 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082606 s, 12.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg405-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg405-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.23895 s, 16.8 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.11747 s, 20.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.392793 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552075 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0729808 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515944 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601558 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178543 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269133 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318715 s, 3.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312124 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230693 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254589 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313938 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277788 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239228 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252267 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363746 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302675 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018584 s, 56.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539516 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215141 s, 48.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220472 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242074 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278007 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266964 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425279 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038071 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06802 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436994 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473013 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103201 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527714 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532493 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553661 s, 18.9 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg405-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg405-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.10475 s, 14.8 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.08761 s, 20.6 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (81s) == sanityn test 77o: Changing rank should not panic ====== 10:55:43 (1761317743) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (10s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 10:55:53 (1761317753) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (96s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 10:57:29 (1761317849) mds.MDS.mdt.nrs_policies=tbf oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg405-client: oleg405-server: ssh exited with exit code 36 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg405-client: oleg405-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (31s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 10:58:00 (1761317880) oleg405-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg405-client: oleg405-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (8s) == sanityn test 79: xattr: intent error ================== 10:58:08 (1761317888) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 10:58:10.000000000 -0400 Modify: 2025-10-24 10:58:10.000000000 -0400 Change: 2025-10-24 10:58:10.000000000 -0400 Birth: 2025-10-24 10:58:10.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (17s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 10:58:25 (1761317905) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 10:58:28 (1761317908) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 10:58:31 (1761317911) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (4s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 10:58:34 (1761317914) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (3s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 10:58:37 (1761317917) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (3s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 10:58:40 (1761317920) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7132 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7132 PASS 82 (7s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 10:58:47 (1761317927) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (2s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 10:58:49 (1761317929) fail_loc=0x8000060b PASS 84 (11s) == sanityn test 90: open/create and unlink striped directory ========================================================== 10:59:00 (1761317940) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (2s) == sanityn test 91: chmod and unlink striped directory === 10:59:03 (1761317943) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) == sanityn test 92: create remote directory under orphan directory ========================================================== 10:59:05 (1761317945) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 10:59:08 (1761317948) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2640 0xa50 0 1 2626 0xa42 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2641 0xa51 0 1 2627 0xa43 0 rc1=0 and rc2=0 PASS 93 (14s) == sanityn test 94: signal vs CP callback race =========== 10:59:21 (1761317961) osc.lustre-OST0000-osc-ffff9b4c870cd000.idle_timeout=0 osc.lustre-OST0000-osc-ffff9b4c99097800.idle_timeout=0 osc.lustre-OST0001-osc-ffff9b4c870cd000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9b4c99097800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.206852 s, 39.6 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.045 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 282860 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0708984 s, 57.8 kB/s osc.lustre-OST0000-osc-ffff9b4c870cd000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9b4c99097800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9b4c870cd000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9b4c99097800.idle_timeout=debug PASS 94 (14s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 10:59:36 (1761317976) llite.lustre-ffff9b4c870cd000.fast_read=0 llite.lustre-ffff9b4c99097800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00557953 s, 2.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff9b4c870cd000.fast_read=1 llite.lustre-ffff9b4c99097800.fast_read=1 PASS 95b (12s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 10:59:47 (1761317987) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 10:59:50 (1761317990) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00450166 s, 910 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (6s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 10:59:55 (1761317995) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0693307 s, 30.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (5s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 11:00:00 (1761318000) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0973185 s, 21.5 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (5s) == sanityn test 100e: DoM: read on open and file size ==== 11:00:05 (1761318005) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00596083 s, 2.0 MB/s PASS 100e (7s) == sanityn test 101a: Discard DoM data on unlink ========= 11:00:12 (1761318012) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000694052 s, 5.9 MB/s PASS 101a (6s) == sanityn test 101b: Discard DoM data on rename ========= 11:00:18 (1761318018) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00067365 s, 6.1 MB/s PASS 101b (6s) == sanityn test 101c: Discard DoM data on close-unlink === 11:00:25 (1761318025) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00762557 s, 537 kB/s PASS 101c (7s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 11:00:34 (1761318034) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1761318037 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1761318040 Finishing test test23 at 1761318040 PASS 103 (12s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 11:00:47 (1761318047) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 11:00:50 (1761318050) fail_loc=0x416 PASS 105 (52s) == sanityn test 106a: Verify the btime via statx() ======= 11:01:42 (1761318102) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (3s) == sanityn test 106b: Glimpse RPCs test for statx ======== 11:01:45 (1761318105) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0106068 s, 96.5 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 11:01:46.000000000 -0400 Modify: 2025-10-24 11:01:46.000000000 -0400 Change: 2025-10-24 11:01:46.000000000 -0400 Birth: 2025-10-24 11:01:47.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1761318107 1761318106 1761318106 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 11:01:46.000000000 -0400 Modify: 2025-10-24 11:01:46.000000000 -0400 Change: 2025-10-24 11:01:46.000000000 -0400 Birth: 2025-10-24 11:01:47.000000000 -0400 1761318106 1024 1 PASS 106b (6s) == sanityn test 106c: Verify statx attributes mask ======= 11:01:51 (1761318111) PASS 106c (6s) == sanityn test 107a: Basic grouplock conflict =========== 11:01:57 (1761318117) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.196169 s, 53.5 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2649 0xa59 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 First grouplock blocks second one PASS 107a (8s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 11:02:05 (1761318125) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.250367 s, 41.9 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2632 0xa48 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (12s) == sanityn test 108a: lseek: parallel updates ============ 11:02:18 (1761318138) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00504823 s, 2.0 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.75895 s, 1.8 MB/s PASS 108a (12s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 11:02:29 (1761318149) 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre 192.168.204.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg405-client.virtnet /mnt/lustre (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: Force libcfs to create 2 CPU partitions Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre Starting client: oleg405-client.virtnet: -o user_xattr,flock oleg405-server@tcp:/lustre /mnt/lustre2 PASS 109 (1092s) == sanityn test 110: do not grant another lock on resend ========================================================== 11:20:41 (1761319241) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 11:20:42.000000000 -0400 Modify: 2025-10-24 11:20:42.000000000 -0400 Change: 2025-10-24 11:20:42.000000000 -0400 Birth: 2025-10-24 11:20:42.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg405-client.virtnet oleg405-client.virtnet PASS 110 (39s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 11:21:20 (1761319280) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (2s) == sanityn test 112: update max-inherit in default LMV === 11:21:22 (1761319282) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (2s) == sanityn test 113: check servers of specified fs ======= 11:21:24 (1761319284) lustre-OST0000-osc-ffff9b4c986d6800 active. lustre-OST0001-osc-ffff9b4c986d6800 active. lustre-OST0000-osc-ffff9b4c986d6800 active. lustre-OST0001-osc-ffff9b4c986d6800 active. MGC192.168.204.105@tcp active. PASS 113 (4s) cleanup: ====================================================== == sanityn test complete, duration 7791 sec ============== 11:21:30 (1761319290) Stopping clients: oleg405-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg405-client.virtnet /mnt/lustre2 opts: