-----============= acceptance-small: sanityn ============----- Wed Jan 7 21:33:40 EST 2026 oleg358-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Started clients oleg358-client.virtnet: 192.168.203.158@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg358-client.virtnet: executing check_config_client /mnt/lustre oleg358-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg358-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9e9dc54a2000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9e9dcb7c1000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9e9dc54a2000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9e9dcb7c1000.idle_timeout=debug disable quota as required oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242952 s, 43.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7847] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 21:34:06 (1767839646) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 21:34:09 (1767839649) -rw-r--r-- 1 root root 0 Jan 7 21:34 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 21:34:13 (1767839653) -rw-r--r-- 1 root root 0 Jan 7 21:34 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 21:34:17 (1767839657) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 21:34:20 (1767839660) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 21:34:24 (1767839664) total 1 -rwxrwxrwx 1 root root 0 Jan 7 21:34 f2a -rwxrwxrwx 1 root root 0 Jan 7 21:34 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 7 21:34 f2a -rwxrwxrwx 1 root root 0 Jan 7 21:34 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0049846 s, 103 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 21:34:28 (1767839668) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (4s) == sanityn test 2g: check blocks update on sync write ==== 21:34:32 (1767839672) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.227604 s, 9.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 21:34:35 (1767839675) PASS 3 (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 21:34:40 (1767839680) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 21:34:45 (1767839685) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (5s) == sanityn test 6: remove of open file on other node ============================================================================ 21:34:50 (1767839690) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 21:34:54 (1767839694) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (4s) == sanityn test 8: remove of open special file on other node ==================================================================== 21:34:58 (1767839698) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 21:35:02 (1767839702) PASS 9a (4s) == sanityn test 9b: append to striped sparse file ======== 21:35:06 (1767839706) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339305 s, 30.9 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0338609 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (4s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 21:35:10 (1767839710) 1+0 records in 1+0 records out 1 byte copied, 0.00380796 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00377764 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.018432 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0109305 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00872172 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00409691 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0120343 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00455229 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00438076 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00407725 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00426727 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00594081 s, 0.2 kB/s PASS 10a (5s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 21:35:15 (1767839715) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0092542 s, 332 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0338514 s, 121 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000220947 s, 13.9 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 21:35:19 (1767839719) striped dir -i1 -c2 -H all_char /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 21:35:24 (1767839724) warning: '-runtime' deprecated, use '-t runtime' instead - link 679 (time 1767839735.68 total 10.01 last 67.83) - link 1330 (time 1767839745.69 total 20.02 last 65.06) - link 1957 (time 1767839755.69 total 30.02 last 62.67) - link 2695 (time 1767839765.70 total 40.03 last 73.74) - link 3331 (time 1767839775.70 total 50.03 last 63.58) - link 3844 (time 1767839785.73 total 60.06 last 51.17) - link 4379 (time 1767839795.73 total 70.06 last 53.49) - link 4848 (time 1767839805.73 total 80.06 last 46.88) - link 5393 (time 1767839815.75 total 90.08 last 54.43) total: 6013 link in 100.01 seconds: 60.13 ops/second using seed 3606295752 running for 100 seconds - stat 10000 (time 1767839733 ; total 7 ; last 7) - stat 20000 (time 1767839739 ; total 13 ; last 6) - stat 30000 (time 1767839746 ; total 20 ; last 7) - stat 40000 (time 1767839752 ; total 26 ; last 6) - stat 50000 (time 1767839758 ; total 32 ; last 6) - stat 60000 (time 1767839764 ; total 38 ; last 6) - stat 70000 (time 1767839770 ; total 44 ; last 6) - stat 80000 (time 1767839776 ; total 50 ; last 6) - stat 87529 (time 1767839787 ; total 61 ; last 11) - stat 90000 (time 1767839790 ; total 64 ; last 3) - stat 97317 (time 1767839801 ; total 75 ; last 11) - stat 100000 (time 1767839805 ; total 79 ; last 4) - stat 108803 (time 1767839816 ; total 90 ; last 11) - stat 110000 (time 1767839817 ; total 91 ; last 1) total: 116721 stats in 100 seconds: 1167.209961 stats/second - unlinked 0 (time 1767839776 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile6013) error: No such file or directory total: 6012 unlinks in 82 seconds: 73.317070 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20960) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20962) - No such process PASS 12 (139s) == sanityn test 13: test directory page revocation ======= 21:37:42 (1767839862) striped dir -i1 -c2 -H all_char /mnt/lustre/d13 f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 21:37:46 (1767839866) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22158 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 21:37:50 (1767839870) striped dir -i0 -c2 -H all_char /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22742 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 21:37:53 (1767839873) striped dir -i0 -c2 -H crush /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23328 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 21:37:56 (1767839876) striped dir -i0 -c2 -H all_char /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23915 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 21:38:00 (1767839880) striped dir -i0 -c2 -H all_char /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24503 Terminated $DIR1/$tdir/sleep 60 PASS 14d (5s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 21:38:05 (1767839885) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7528448 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7528448kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (15s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 21:38:20 (1767839900) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.249562 s, 8.4 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.166727 s, 12.6 MB/s Chance of close/open is 1 in 50 Seed set to 9902 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000004[0] 1767839902.991581 write 0x606524 thru 0x6164a9 (0xff86 bytes) 000006[0] 1767839903.007759 mapwrite 0x3e7b36 thru 0x3ea98f (0x2e5a bytes) truncating to largest ever: 0x26976 000007[0] 1767839903.093817 trunc from 0x6164aa to 0x026975 (0x5efb34 bytes) 000009[0] 1767839903.150991 write 0x1a9a6d thru 0x1adbd6 (0x416a bytes) 000011[0] 1767839903.153623 mapwrite 0x12a51e thru 0x13160b (0x70ee bytes) 000012[1] 1767839903.218731 mapread 0x117e94 thru 0x120c52 (0x8dbf bytes) 000013[0] 1767839903.278893 mapwrite 0x057bef thru 0x0593d5 (0x17e7 bytes) 000014[0] 1767839903.331597 read 0x00d7b9 thru 0x016b65 (0x93ad bytes) 000015[1] 1767839903.340981 read 0x19d765 thru 0x1ad561 (0xfdfd bytes) 000017[1] 1767839903.351574 mapwrite 0x7f045a thru 0x7f62f9 (0x5ea0 bytes) 000019[1] 1767839903.475720 read 0x3d7fd1 thru 0x3e2ec7 (0xaef7 bytes) 000020[1] 1767839903.535888 read 0x70b590 thru 0x719f59 (0xe9ca bytes) 000022[0] 1767839903.587323 mapread 0x58ee38 thru 0x594364 (0x552d bytes) 000025[0] 1767839903.645552 write 0x9d5840 thru 0x9db057 (0x5818 bytes) 000026[1] 1767839903.665482 mapwrite 0x2b3686 thru 0x2b686d (0x31e8 bytes) 000027[0] 1767839903.725294 read 0x867134 thru 0x86e363 (0x7230 bytes) 000029[0] 1767839903.762362 mapread 0x1e3d2d thru 0x1ea963 (0x6c37 bytes) 000030[0] 1767839903.804760 mapread 0x2e0a7e thru 0x2e6c47 (0x61ca bytes) 000032[1] 1767839903.854420 write 0x2a572e thru 0x2af378 (0x9c4b bytes) 000033[0] 1767839903.882395 write 0x70577e thru 0x709835 (0x40b8 bytes) 000035[1] 1767839903.920793 mapread 0x2690f6 thru 0x27422d (0xb138 bytes) 000037[1] 1767839903.986394 mapread 0x98c950 thru 0x998633 (0xbce4 bytes) 000038[0] 1767839904.059689 mapread 0x2a654a thru 0x2b581c (0xf2d3 bytes) 000042[0] 1767839904.153078 mapwrite 0x66a77d thru 0x6779d4 (0xd258 bytes) 000043[0] 1767839904.285114 write 0x702cab thru 0x7042da (0x1630 bytes) 000044[1] 1767839904.311072 read 0x5670f0 thru 0x574a32 (0xd943 bytes) 000045[1] 1767839904.345641 mapwrite 0x3ef5e0 thru 0x3f5f68 (0x6989 bytes) 000046[1] 1767839904.449694 read 0x70b9a1 thru 0x719395 (0xd9f5 bytes) truncating to largest ever: 0x73deb7 000048[1] 1767839904.507122 trunc from 0x9db058 to 0x73deb6 (0x29d1a1 bytes) 000052[1] 1767839904.596584 read 0x167d42 thru 0x168d39 (0xff8 bytes) 000057[0] 1767839904.692088 mapwrite 0x2e8e53 thru 0x2f17c7 (0x8975 bytes) 000058[0] 1767839904.816394 trunc from 0x73deb7 to 0x1c43f0 (0x579ac6 bytes) 000064[0] 1767839904.898478 write 0x57406c thru 0x57d952 (0x98e7 bytes) 000065[0] 1767839904.920598 write 0x693c6b thru 0x697b34 (0x3eca bytes) 000067[1] 1767839904.938636 mapwrite 0x34f901 thru 0x3597f2 (0x9ef2 bytes) 000068[1] 1767839905.038995 mapread 0x577de6 thru 0x57bb08 (0x3d23 bytes) 000069[1] 1767839905.072505 trunc from 0x697b35 to 0x61e65c (0x794d8 bytes) truncating to largest ever: 0x7af298 000070[0] 1767839905.111174 trunc from 0x61e65d to 0x7af297 (0x190c3b bytes) 000073[0] 1767839905.140792 trunc from 0x7af298 to 0x1e6586 (0x5c8d11 bytes) 000074[1] 1767839905.161417 trunc from 0x1e6587 to 0x3a45a6 (0x1be020 bytes) 000078[1] 1767839905.252620 write 0x5e85e8 thru 0x5f359b (0xafb4 bytes) 000079[0] 1767839905.274419 mapwrite 0x1dec0f thru 0x1e44d8 (0x58ca bytes) 000082[1] 1767839905.360571 trunc from 0x5f359c to 0x3aeca5 (0x2448f6 bytes) 000084[1] 1767839905.425428 trunc from 0x3aeca6 to 0x7187dc (0x369b37 bytes) 000087[0] 1767839905.472026 trunc from 0x7187dd to 0x0cd350 (0x64b48c bytes) 000088[0] 1767839905.529539 write 0x1ab94c thru 0x1b4070 (0x8725 bytes) 000089[0] 1767839905.550499 read 0x06bbe1 thru 0x0729f2 (0x6e12 bytes) 000091[1] 1767839905.614195 trunc from 0x1b4071 to 0x565899 (0x3b1829 bytes) 000092[0] 1767839905.658753 write 0x70264c thru 0x70b6f1 (0x90a6 bytes) 000093[1] 1767839905.672637 mapread 0x4ccada thru 0x4d7a08 (0xaf2f bytes) 000094[0] 1767839905.761348 mapread 0x1d33e3 thru 0x1da83a (0x7458 bytes) 000099[0] 1767839905.837890 mapwrite 0x8348d9 thru 0x843f01 (0xf629 bytes) 000100[0] 1767839905.960899 mapread 0x085c90 thru 0x08f83d (0x9bae bytes) 000101[1] 1767839906.032552 trunc from 0x843f02 to 0x436769 (0x40d798 bytes) 000104[1] 1767839906.075326 write 0x96535a thru 0x96f61d (0xa2c4 bytes) 000105[0] 1767839906.077415 mapread 0x5aaf09 thru 0x5b546c (0xa564 bytes) 000107[0] 1767839906.138762 read 0x5e3d5d thru 0x5ef0bb (0xb35f bytes) 000108[1] 1767839906.144067 write 0x03dabe thru 0x03eaa6 (0xfe9 bytes) 000109[0] 1767839906.170416 mapread 0x229616 thru 0x23516d (0xbb58 bytes) 000110[1] 1767839906.251513 trunc from 0x96f61e to 0x485df9 (0x4e9824 bytes) 000115[1] 1767839906.363888 mapread 0x1e4f1a thru 0x1edaf2 (0x8bd9 bytes) 000117[0] 1767839906.421490 write 0x2bda42 thru 0x2c1952 (0x3f11 bytes) 000118[0] 1767839906.443359 trunc from 0x485dfa to 0x6099c9 (0x183bd0 bytes) truncating to largest ever: 0x9da8b4 000119[1] 1767839906.474474 trunc from 0x6099ca to 0x9da8b3 (0x3d0eea bytes) 000120[0] 1767839906.537262 mapwrite 0x75e377 thru 0x768db6 (0xaa40 bytes) 000122[0] 1767839906.635933 read 0x0a69f5 thru 0x0b21dd (0xb7e9 bytes) 000123[1] 1767839906.748499 read 0x3e4b96 thru 0x3ef02b (0xa496 bytes) 000126[0] 1767839906.818691 mapwrite 0x02f2ee thru 0x03715d (0x7e70 bytes) 000127[1] 1767839906.887373 write 0x768aed thru 0x76edf2 (0x6306 bytes) 000128[0] 1767839906.908369 write 0x46f8ff thru 0x47d6cf (0xddd1 bytes) 000129[0] 1767839906.925740 write 0x1724c3 thru 0x17aaa7 (0x85e5 bytes) 000130[0] 1767839906.967092 trunc from 0x9da8b4 to 0x053637 (0x98727c bytes) 000131[1] 1767839907.004725 write 0x43cd62 thru 0x44c9a0 (0xfc3f bytes) 000132[1] 1767839907.019687 mapread 0x126b99 thru 0x126bc5 (0x2d bytes) 000133[1] 1767839907.035641 mapread 0x227888 thru 0x2293d5 (0x1b4e bytes) 000135[1] 1767839907.072420 trunc from 0x44c9a1 to 0x015de9 (0x436bb7 bytes) 000136[0] 1767839907.092680 mapwrite 0x0e195c thru 0x0e7687 (0x5d2c bytes) 000141[0] 1767839907.215519 write 0x052634 thru 0x05f0bb (0xca88 bytes) 000142[0] 1767839907.236584 read 0x03bacc thru 0x040845 (0x4d7a bytes) 000143[1] 1767839907.248784 mapwrite 0x8ee150 thru 0x8fde04 (0xfcb5 bytes) 000144[0] 1767839907.377957 trunc from 0x8fde05 to 0x1fb20b (0x702bf9 bytes) 000145[1] 1767839907.395777 mapread 0x076ec9 thru 0x085e69 (0xefa1 bytes) 000146[0] 1767839907.465871 mapwrite 0x085bbc thru 0x090d9d (0xb1e2 bytes) 000147[0] 1767839907.592043 write 0x34e030 thru 0x35716c (0x913d bytes) 000148[0] 1767839907.593514 trunc from 0x35716d to 0x156749 (0x200a23 bytes) 000150[1] 1767839907.615935 mapread 0x0a6024 thru 0x0b1c99 (0xbc76 bytes) 000151[1] 1767839907.676100 mapwrite 0x40f28e thru 0x41012e (0xea1 bytes) 000152[0] 1767839907.754537 mapwrite 0x67398a thru 0x682b5b (0xf1d2 bytes) 000154[1] 1767839907.869072 write 0x7ea17c thru 0x7f5421 (0xb2a6 bytes) 000155[1] 1767839907.885622 trunc from 0x7f5422 to 0x94b63a (0x156219 bytes) 000156[1] 1767839907.940425 write 0x35f94e thru 0x36cab7 (0xd16a bytes) 000158[0] 1767839907.951502 trunc from 0x94b63b to 0x530921 (0x41ad19 bytes) 000159[1] 1767839908.000514 write 0x2efb8a thru 0x2f14a8 (0x191f bytes) 000160[0] 1767839908.039534 mapwrite 0x0a6a6e thru 0x0af350 (0x88e3 bytes) 000161[0] 1767839908.189221 mapwrite 0x0f0675 thru 0x0f785a (0x71e6 bytes) 000163[0] 1767839908.335468 mapread 0x15d51b thru 0x1632a5 (0x5d8b bytes) 000165[1] 1767839908.375992 read 0x130b34 thru 0x13e8bb (0xdd88 bytes) 000167[0] 1767839908.427643 mapread 0x48e7db thru 0x499295 (0xaabb bytes) 000169[0] 1767839908.485030 write 0x6d7682 thru 0x6d7f56 (0x8d5 bytes) 000170[1] 1767839908.486009 read 0x4b08b3 thru 0x4b77dc (0x6f2a bytes) 000171[1] 1767839908.526309 trunc from 0x6d7f57 to 0x27a301 (0x45dc55 bytes) 000172[0] 1767839908.562561 mapwrite 0x04b8da thru 0x05566a (0x9d91 bytes) 000173[0] 1767839908.653753 mapwrite 0x34af8b thru 0x34bca4 (0xd1a bytes) 000175[1] 1767839908.786623 read 0x0858d1 thru 0x08a758 (0x4e88 bytes) 000176[0] 1767839908.828499 trunc from 0x34bca5 to 0x13e665 (0x20d63f bytes) 000177[0] 1767839908.872325 read 0x05817e thru 0x05b2c4 (0x3147 bytes) 000180[1] 1767839908.923115 read 0x098116 thru 0x099418 (0x1303 bytes) 000181[0] 1767839908.942618 write 0x140417 thru 0x144ea4 (0x4a8e bytes) 000184[1] 1767839908.962813 write 0x177a1d thru 0x17d878 (0x5e5c bytes) 000186[1] 1767839908.993480 read 0x066e75 thru 0x06b3c9 (0x4555 bytes) 000193[1] 1767839909.061789 mapread 0x028d79 thru 0x029ccc (0xf54 bytes) 000194[0] 1767839909.068284 mapread 0x15a1de thru 0x15fe26 (0x5c49 bytes) 000196[1] 1767839909.101802 mapread 0x0494b3 thru 0x050bde (0x772c bytes) 000198[0] 1767839909.127347 mapwrite 0x27de6b thru 0x28ca9a (0xec30 bytes) 000199[0] 1767839909.261914 trunc from 0x28ca9b to 0x5bc264 (0x32f7ca bytes) 000201[1] 1767839909.277306 mapwrite 0x01f0b5 thru 0x02aef7 (0xbe43 bytes) 000203[1] 1767839909.427772 read 0x307994 thru 0x310caf (0x931c bytes) 000205[1] 1767839909.490404 read 0x4226d6 thru 0x4241fb (0x1b26 bytes) 000207[0] 1767839909.509091 mapread 0x3f0618 thru 0x3ffacb (0xf4b4 bytes) 000208[1] 1767839909.593511 mapread 0x17575f thru 0x184aca (0xf36c bytes) 000209[0] 1767839909.630859 read 0x1ac916 thru 0x1b31c6 (0x68b1 bytes) 000211[1] 1767839909.650251 mapread 0x1a887f thru 0x1b18a4 (0x9026 bytes) 000216[1] 1767839909.717490 mapread 0x44700c thru 0x44ced3 (0x5ec8 bytes) 000218[1] 1767839909.726105 read 0x11dea6 thru 0x1262f8 (0x8453 bytes) 000222[1] 1767839909.776002 write 0x43c590 thru 0x4486e3 (0xc154 bytes) 000224[1] 1767839909.777201 mapwrite 0x1c052c thru 0x1cc98d (0xc462 bytes) 000225[0] 1767839909.886746 mapread 0x07412a thru 0x078769 (0x4640 bytes) 000226[0] 1767839909.949073 write 0x152382 thru 0x15e71c (0xc39b bytes) 000227[0] 1767839909.997943 mapread 0x151748 thru 0x160ee7 (0xf7a0 bytes) 000228[1] 1767839910.013079 mapwrite 0x235cee thru 0x23b64a (0x595d bytes) 000229[0] 1767839910.103584 mapwrite 0x2c0d18 thru 0x2cea65 (0xdd4e bytes) 000230[1] 1767839910.238581 mapwrite 0x116000 thru 0x11870e (0x270f bytes) 000232[1] 1767839910.334096 mapwrite 0x720041 thru 0x728b49 (0x8b09 bytes) 000236[0] 1767839910.445113 write 0x91ca96 thru 0x91d286 (0x7f1 bytes) 000237[0] 1767839910.467093 write 0x37731d thru 0x379d43 (0x2a27 bytes) 000240[0] 1767839910.537547 mapwrite 0x585442 thru 0x5908d8 (0xb497 bytes) 000241[1] 1767839910.600496 mapread 0x64f290 thru 0x659e4b (0xabbc bytes) 000242[0] 1767839910.656603 mapread 0x823738 thru 0x82fb7f (0xc448 bytes) 000243[0] 1767839910.689368 mapwrite 0x94b891 thru 0x9522fd (0x6a6d bytes) 000249[1] 1767839910.784476 mapread 0x1707d6 thru 0x171559 (0xd84 bytes) 000250[1] 1767839910.821008 mapwrite 0x5bd4be thru 0x5cbc31 (0xe774 bytes) 000252[1] 1767839911.000786 read 0x271f1c thru 0x27c27d (0xa362 bytes) 000254[1] 1767839911.079641 mapread 0x2e191b thru 0x2ee404 (0xcaea bytes) 000255[1] 1767839911.095844 mapread 0x944be4 thru 0x9522fd (0xd71a bytes) 000258[1] 1767839911.122566 mapread 0x56a9aa thru 0x56bcc6 (0x131d bytes) 000259[1] 1767839911.139589 write 0x9480ad thru 0x9540a0 (0xbff4 bytes) 000260[0] 1767839911.140685 read 0x35d612 thru 0x35e79f (0x118e bytes) 000263[0] 1767839911.156015 mapread 0x89f50f thru 0x8a41ef (0x4ce1 bytes) 000266[0] 1767839911.166767 mapread 0x702cc9 thru 0x7105c8 (0xd900 bytes) 000269[0] 1767839911.178857 mapread 0x7810bc thru 0x78a5ab (0x94f0 bytes) 000270[0] 1767839911.196174 write 0x50349b thru 0x50429d (0xe03 bytes) 000273[1] 1767839911.224661 mapread 0x8eea0f thru 0x8fa386 (0xb978 bytes) 000275[1] 1767839911.234478 read 0x942924 thru 0x94b476 (0x8b53 bytes) 000279[1] 1767839911.259059 read 0x36180d thru 0x36be00 (0xa5f4 bytes) 000282[1] 1767839911.286403 trunc from 0x9540a1 to 0x7b9c53 (0x19a44d bytes) 000283[1] 1767839911.336981 read 0x4a1cfd thru 0x4afa8d (0xdd91 bytes) 000284[0] 1767839911.400514 mapwrite 0x0a11e3 thru 0x0b113f (0xff5d bytes) 000287[0] 1767839911.528515 trunc from 0x7b9c54 to 0x1638eb (0x656368 bytes) 000288[0] 1767839911.555554 write 0x9c0383 thru 0x9c15e2 (0x1260 bytes) 000292[1] 1767839911.596107 mapwrite 0x216759 thru 0x219592 (0x2e3a bytes) 000293[1] 1767839911.742107 mapread 0x41f4ae thru 0x42b7d3 (0xc326 bytes) 000294[0] 1767839911.880351 trunc from 0x9c15e3 to 0x0409ed (0x980bf5 bytes) 000296[1] 1767839911.928297 write 0x96fb28 thru 0x976ed7 (0x73b0 bytes) 000298[1] 1767839911.966996 write 0x7f129a thru 0x7ff928 (0xe68f bytes) 000299[0] 1767839911.992488 trunc from 0x976ed8 to 0x8a4bff (0xd22d8 bytes) 000300[1] 1767839912.028883 mapread 0x4e9829 thru 0x4f8095 (0xe86d bytes) 000304[0] 1767839912.200602 mapread 0x47134e thru 0x47611a (0x4dcd bytes) 000312[1] 1767839912.303323 mapread 0x697c76 thru 0x6a3da3 (0xc12e bytes) 000315[1] 1767839912.370263 trunc from 0x8a4c00 to 0x8b4348 (0xf749 bytes) 000317[1] 1767839912.395561 write 0x4a86fc thru 0x4a8b93 (0x498 bytes) 000318[1] 1767839912.426830 read 0x2f34b9 thru 0x2fd38f (0x9ed7 bytes) 000320[1] 1767839912.508542 mapwrite 0x850e0b thru 0x85618a (0x5380 bytes) 000321[0] 1767839912.584777 trunc from 0x8b4349 to 0x860f72 (0x533d6 bytes) 000326[1] 1767839912.676896 trunc from 0x860f73 to 0x8b3585 (0x52613 bytes) 000327[1] 1767839912.738220 mapwrite 0x55785d thru 0x562da7 (0xb54b bytes) 000328[1] 1767839912.904799 mapread 0x7372c2 thru 0x74611f (0xee5e bytes) 000329[1] 1767839912.985697 mapread 0x04697d thru 0x048dc2 (0x2446 bytes) 000331[1] 1767839912.995915 mapread 0x0528b0 thru 0x05c4da (0x9c2b bytes) 000332[1] 1767839913.052822 trunc from 0x8b3586 to 0x917175 (0x63bf0 bytes) 000333[1] 1767839913.078055 write 0x7f9817 thru 0x80558f (0xbd79 bytes) 000334[0] 1767839913.105901 mapwrite 0x71082e thru 0x7159b0 (0x5183 bytes) 000337[0] 1767839913.222747 read 0x4557dd thru 0x45af45 (0x5769 bytes) 000339[1] 1767839913.301480 trunc from 0x917176 to 0x8cdadb (0x4969a bytes) 000341[0] 1767839913.373698 trunc from 0x8cdadc to 0x4dc630 (0x3f14ab bytes) 000343[1] 1767839913.447835 mapwrite 0x1dc271 thru 0x1ec02b (0xfdbb bytes) 000346[1] 1767839913.687244 read 0x24a52e thru 0x2547a1 (0xa274 bytes) 000347[1] 1767839913.736835 mapwrite 0x59f73e thru 0x5a1db2 (0x2675 bytes) 000350[0] 1767839913.878833 read 0x33f22b thru 0x345dc6 (0x6b9c bytes) 000351[0] 1767839913.925047 trunc from 0x5a1db3 to 0x0dfbff (0x4c21b3 bytes) 000352[0] 1767839913.994587 read 0x098dd8 thru 0x0a21cd (0x93f6 bytes) 000353[1] 1767839914.018761 read 0x077eff thru 0x08393e (0xba40 bytes) 000355[1] 1767839914.068590 mapread 0x07966d thru 0x088cb2 (0xf646 bytes) 000359[0] 1767839914.108223 write 0x565c91 thru 0x56fa32 (0x9da2 bytes) 000362[1] 1767839914.124722 write 0x51b5c4 thru 0x52709f (0xbadc bytes) 000367[1] 1767839914.156599 mapread 0x0af210 thru 0x0b6f70 (0x7d61 bytes) 000368[0] 1767839914.170747 trunc from 0x56fa33 to 0x868741 (0x2f8d0f bytes) 000369[1] 1767839914.201053 read 0x6b449e thru 0x6b80a6 (0x3c09 bytes) 000373[0] 1767839914.250052 trunc from 0x868742 to 0x7d4b8f (0x93bb2 bytes) 000375[1] 1767839914.315615 mapwrite 0x2c8b0e thru 0x2d6bdb (0xe0ce bytes) 000376[0] 1767839914.498455 mapread 0x5ad59a thru 0x5b787e (0xa2e5 bytes) 000378[0] 1767839914.569817 write 0x13f635 thru 0x141542 (0x1f0e bytes) 000380[1] 1767839914.603480 write 0x052828 thru 0x060208 (0xd9e1 bytes) 000381[1] 1767839914.615881 mapwrite 0x03f257 thru 0x045ed3 (0x6c7d bytes) 000382[1] 1767839914.701265 read 0x59c45d thru 0x5ac2a6 (0xfe4a bytes) 000383[1] 1767839914.759125 mapread 0x05ae8d thru 0x05b666 (0x7da bytes) 000384[0] 1767839914.780765 read 0x76cd30 thru 0x7758f7 (0x8bc8 bytes) 000385[1] 1767839914.812966 trunc from 0x7d4b90 to 0x00887b (0x7cc314 bytes) 000386[1] 1767839914.851570 mapwrite 0x5f8aae thru 0x601a0f (0x8f62 bytes) 000391[0] 1767839915.051585 trunc from 0x601a10 to 0x72843d (0x126a2e bytes) 000392[1] 1767839915.095815 mapwrite 0x7da9f1 thru 0x7e2951 (0x7f61 bytes) 000393[0] 1767839915.190684 mapread 0x154046 thru 0x1628ad (0xe868 bytes) 000394[1] 1767839915.277711 mapwrite 0x1fce12 thru 0x203f60 (0x714f bytes) 000395[1] 1767839915.396232 write 0x448525 thru 0x44f1ce (0x6caa bytes) 000396[1] 1767839915.418700 write 0x806282 thru 0x80da9e (0x781d bytes) 000398[1] 1767839915.420607 mapread 0x0cb682 thru 0x0d6a25 (0xb3a4 bytes) 000401[1] 1767839915.534829 mapread 0x5d47bb thru 0x5e1064 (0xc8aa bytes) 000402[0] 1767839915.587826 mapwrite 0x7e77c6 thru 0x7ec5d9 (0x4e14 bytes) 000403[0] 1767839915.769634 trunc from 0x80da9f to 0x343dff (0x4c9c9f bytes) 000405[0] 1767839915.805843 mapread 0x1e6cc9 thru 0x1ebc13 (0x4f4b bytes) 000407[1] 1767839915.849215 mapwrite 0x4fd8c2 thru 0x504d13 (0x7452 bytes) 000408[1] 1767839916.031785 mapwrite 0x1a3480 thru 0x1b1ef2 (0xea73 bytes) 000409[0] 1767839916.130100 mapwrite 0x2f81dc thru 0x303730 (0xb555 bytes) 000410[0] 1767839916.255448 trunc from 0x504d14 to 0x0a5fe6 (0x45ed2d bytes) 000412[0] 1767839916.282615 read 0x098c5b thru 0x09fbba (0x6f60 bytes) 000413[0] 1767839916.288677 mapwrite 0x837f88 thru 0x83b467 (0x34e0 bytes) 000415[0] 1767839916.443504 trunc from 0x83b468 to 0x651fd0 (0x1e9497 bytes) 000416[0] 1767839916.466991 mapread 0x4b932e thru 0x4bd085 (0x3d58 bytes) 000420[0] 1767839916.479742 mapwrite 0x409d1b thru 0x409ec7 (0x1ad bytes) 000422[0] 1767839916.547349 write 0x609ec0 thru 0x616992 (0xcad3 bytes) 000423[1] 1767839916.548409 mapread 0x24195f thru 0x250530 (0xebd2 bytes) 000424[0] 1767839916.648774 trunc from 0x651fd1 to 0x0ffdf2 (0x5521de bytes) 000427[1] 1767839916.690095 trunc from 0x0ffdf3 to 0x4dc011 (0x3dc21f bytes) 000432[1] 1767839916.749584 trunc from 0x4dc012 to 0x2b7159 (0x224eb8 bytes) 000436[0] 1767839916.845086 mapwrite 0x1b67b3 thru 0x1c138a (0xabd8 bytes) 000441[1] 1767839917.078818 mapread 0x265a75 thru 0x26d0f8 (0x7684 bytes) 000442[0] 1767839917.153600 mapwrite 0x55f9e1 thru 0x56d87d (0xde9d bytes) 000443[1] 1767839917.341541 read 0x0b3394 thru 0x0b5701 (0x236e bytes) 000444[1] 1767839917.374175 read 0x0d32bf thru 0x0dbec1 (0x8c03 bytes) 000446[0] 1767839917.394698 trunc from 0x56d87e to 0x1e2c8e (0x38abef bytes) 000448[1] 1767839917.426821 mapwrite 0x8302ef thru 0x830a03 (0x715 bytes) 000449[1] 1767839917.494408 trunc from 0x830a04 to 0x9a0ee4 (0x1704e1 bytes) 000450[0] 1767839917.503445 read 0x5ae491 thru 0x5bd1bc (0xed2c bytes) 000451[0] 1767839917.534253 mapread 0x099cc1 thru 0x09ef44 (0x5284 bytes) 000452[1] 1767839917.600448 mapwrite 0x367734 thru 0x377144 (0xfa11 bytes) 000453[0] 1767839917.742479 mapread 0x8b0034 thru 0x8b9e2e (0x9dfb bytes) 000454[0] 1767839917.754962 trunc from 0x9a0ee5 to 0x43ee65 (0x56207f bytes) 000455[0] 1767839917.795258 mapread 0x1b1db2 thru 0x1bed52 (0xcfa1 bytes) 000456[1] 1767839917.857110 trunc from 0x43ee66 to 0x5285dc (0xe9777 bytes) 000458[1] 1767839917.884013 mapread 0x4668bb thru 0x46c17b (0x58c1 bytes) 000460[0] 1767839917.926939 mapwrite 0x120224 thru 0x12f7bc (0xf599 bytes) 000461[1] 1767839918.103026 mapwrite 0x50a182 thru 0x50aecd (0xd4c bytes) 000462[1] 1767839918.159980 write 0x9c74db thru 0x9d35b4 (0xc0da bytes) 000466[0] 1767839918.189878 trunc from 0x9d35b5 to 0x1de83f (0x7f4d75 bytes) 000467[0] 1767839918.231741 mapread 0x1d51bd thru 0x1dd468 (0x82ac bytes) 000471[0] 1767839918.315370 mapread 0x1b40bd thru 0x1c3765 (0xf6a9 bytes) 000476[1] 1767839918.404552 mapread 0x19c3b7 thru 0x1a3b6c (0x77b6 bytes) 000478[0] 1767839918.460519 write 0x70de73 thru 0x7191ce (0xb35c bytes) 000482[1] 1767839918.524076 read 0x25d925 thru 0x25e5d6 (0xcb2 bytes) 000485[0] 1767839918.562939 write 0x082fa0 thru 0x086407 (0x3468 bytes) 000486[0] 1767839918.598000 mapwrite 0x001733 thru 0x005b3c (0x440a bytes) 000487[1] 1767839918.663271 read 0x183639 thru 0x18b25e (0x7c26 bytes) 000488[0] 1767839918.695551 mapread 0x41d0ca thru 0x424277 (0x71ae bytes) 000489[1] 1767839918.736652 read 0x6f1be5 thru 0x6f6131 (0x454d bytes) 000492[1] 1767839918.759469 trunc from 0x7191cf to 0x688360 (0x90e6e bytes) 000493[1] 1767839918.798980 mapread 0x08ab5c thru 0x0943c4 (0x9869 bytes) 000494[0] 1767839918.855726 read 0x61b3f7 thru 0x62b213 (0xfe1d bytes) 000496[0] 1767839918.903747 write 0x5d78bd thru 0x5dc6a2 (0x4de6 bytes) 000497[1] 1767839918.945001 trunc from 0x688361 to 0x500c1b (0x187745 bytes) 000499[0] 1767839919.004190 read 0x2b7418 thru 0x2c6a91 (0xf67a bytes) 000500[0] 1767839919.029460 write 0x76c586 thru 0x77c33c (0xfdb7 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9919 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x2fcfea 000004[0] 1767839919.510413 trunc from 00000000 to 0x2fcfe9 (0x2fcfea bytes) 000006[1] 1767839919.552432 read 0x255000 thru 0x261821 (0xc822 bytes) 000007[0] 1767839919.560684 mapread 0x289000 thru 0x290c08 (0x7c09 bytes) truncating to largest ever: 0x9378b5 000009[0] 1767839919.581054 trunc from 0x2fcfea to 0x9378b4 (0x63a8cb bytes) 000011[0] 1767839919.613510 read 0x524000 thru 0x52dc31 (0x9c32 bytes) 000013[0] 1767839919.648575 mapwrite 0x1d2000 thru 0x1d40f8 (0x20f9 bytes) truncating to largest ever: 0x977749 000019[0] 1767839919.804778 trunc from 0x9378b5 to 0x977748 (0x3fe94 bytes) 000020[1] 1767839919.841200 write 0x0f0000 thru 0x0fae43 (0xae44 bytes) 000023[1] 1767839919.882520 mapwrite 0x760000 thru 0x76035b (0x35c bytes) 000025[0] 1767839919.930402 mapwrite 0x0f3000 thru 0x0fb311 (0x8312 bytes) 000026[0] 1767839919.963566 mapread 0x42a000 thru 0x433d00 (0x9d01 bytes) 000031[0] 1767839920.024739 read 0x6f9000 thru 0x70480e (0xb80f bytes) 000032[0] 1767839920.054219 mapwrite 0x2e2000 thru 0x2e7720 (0x5721 bytes) 000033[1] 1767839920.123557 read 0x855000 thru 0x8588bb (0x38bc bytes) 000034[0] 1767839920.160658 read 0x679000 thru 0x687a9f (0xeaa0 bytes) 000035[0] 1767839920.198791 mapread 0x2f9000 thru 0x307633 (0xe634 bytes) 000036[1] 1767839920.240558 mapwrite 0x434000 thru 0x43f2a9 (0xb2aa bytes) 000039[1] 1767839920.327786 mapwrite 0x260000 thru 0x26dc3c (0xdc3d bytes) 000041[1] 1767839920.443200 trunc from 0x977749 to 0x033518 (0x944230 bytes) 000043[0] 1767839920.478410 mapwrite 0x304000 thru 0x310278 (0xc279 bytes) 000045[0] 1767839920.596239 read 0x28f000 thru 0x295eb3 (0x6eb4 bytes) 000046[0] 1767839920.619684 write 0x6a9000 thru 0x6aa903 (0x1904 bytes) 000048[0] 1767839920.642705 write 0x480000 thru 0x48abe7 (0xabe8 bytes) 000050[1] 1767839920.688464 trunc from 0x6aa904 to 0x0e08aa (0x5ca059 bytes) 000051[1] 1767839920.727830 write 0x9a1000 thru 0x9a5e6f (0x4e70 bytes) 000054[0] 1767839920.760263 mapwrite 0x2bf000 thru 0x2c4bfd (0x5bfe bytes) 000056[1] 1767839920.903891 trunc from 0x9a5e70 to 0x4e278d (0x4c36e2 bytes) 000057[0] 1767839920.926106 trunc from 0x4e278e to 0x5a07a8 (0xbe01b bytes) 000058[1] 1767839920.955263 trunc from 0x5a07a9 to 0x6d257a (0x131dd2 bytes) 000061[0] 1767839920.977635 read 0x3d9000 thru 0x3dcf65 (0x3f66 bytes) 000063[1] 1767839921.018030 mapwrite 0x51d000 thru 0x51d12e (0x12f bytes) 000066[1] 1767839921.066685 trunc from 0x6d257b to 0x217aa8 (0x4baad2 bytes) 000071[0] 1767839921.108123 mapread 0x016000 thru 0x01c63d (0x663e bytes) 000072[1] 1767839921.155302 mapread 0x031000 thru 0x0359de (0x49df bytes) 000074[1] 1767839921.175019 mapwrite 0x5c2000 thru 0x5ce18d (0xc18e bytes) 000075[0] 1767839921.276858 mapwrite 0x580000 thru 0x582a09 (0x2a0a bytes) 000076[1] 1767839921.358208 trunc from 0x5ce18e to 0x67c1f0 (0xae063 bytes) 000077[1] 1767839921.383640 mapwrite 0x350000 thru 0x35a548 (0xa549 bytes) 000078[0] 1767839921.479964 mapread 0x160000 thru 0x16cbbf (0xcbc0 bytes) 000079[1] 1767839921.545026 mapwrite 0x9e3000 thru 0x9f015a (0xd15b bytes) 000081[1] 1767839921.630901 trunc from 0x9f015b to 0x048d6d (0x9a73ed bytes) 000082[1] 1767839921.671282 read 0x020000 thru 0x020dd9 (0xdda bytes) 000083[1] 1767839921.695254 mapwrite 0x7c8000 thru 0x7c879a (0x79b bytes) 000084[1] 1767839921.828111 read 0x6e0000 thru 0x6e6263 (0x6264 bytes) 000087[1] 1767839921.846843 write 0x782000 thru 0x78651c (0x451d bytes) 000088[1] 1767839921.860599 mapwrite 0x63e000 thru 0x644098 (0x6099 bytes) 000089[1] 1767839921.890452 trunc from 0x7c879b to 0x90e987 (0x1461ed bytes) 000090[0] 1767839921.901563 mapwrite 0x028000 thru 0x02d81f (0x5820 bytes) 000092[1] 1767839922.023991 write 0x783000 thru 0x7920b7 (0xf0b8 bytes) 000093[1] 1767839922.047701 mapread 0x529000 thru 0x533e64 (0xae65 bytes) 000094[1] 1767839922.101606 mapwrite 0x030000 thru 0x033038 (0x3039 bytes) 000097[1] 1767839922.179376 mapread 0x409000 thru 0x40e0ca (0x50cb bytes) 000098[1] 1767839922.229304 read 0x732000 thru 0x732ed2 (0xed3 bytes) 000101[1] 1767839922.308084 mapread 0x25f000 thru 0x265029 (0x602a bytes) 000104[0] 1767839922.390698 read 0x813000 thru 0x81adb2 (0x7db3 bytes) 000105[0] 1767839922.424286 mapread 0x389000 thru 0x38cdf9 (0x3dfa bytes) 000106[0] 1767839922.455228 trunc from 0x90e988 to 0x1fae85 (0x713b02 bytes) 000107[0] 1767839922.488443 mapread 0x120000 thru 0x12a923 (0xa924 bytes) 000109[1] 1767839922.550468 write 0x0de000 thru 0x0e627b (0x827c bytes) 000110[0] 1767839922.576419 mapwrite 0x93b000 thru 0x93b5d5 (0x5d6 bytes) 000113[1] 1767839922.705406 mapwrite 0x243000 thru 0x24930a (0x630b bytes) 000114[0] 1767839922.796575 write 0x72e000 thru 0x739345 (0xb346 bytes) 000115[0] 1767839922.814881 read 0x571000 thru 0x573091 (0x2092 bytes) 000116[0] 1767839922.866184 mapread 0x0b6000 thru 0x0b855c (0x255d bytes) 000117[0] 1767839922.888972 read 0x658000 thru 0x664cf1 (0xccf2 bytes) 000121[0] 1767839922.919512 mapread 0x77a000 thru 0x77d59a (0x359b bytes) 000122[0] 1767839922.934228 mapwrite 0x942000 thru 0x95082e (0xe82f bytes) 000123[1] 1767839923.020804 write 0x2c5000 thru 0x2c6f26 (0x1f27 bytes) 000126[1] 1767839923.059097 read 0x93f000 thru 0x946d7c (0x7d7d bytes) 000128[0] 1767839923.088311 read 0x58e000 thru 0x59353c (0x553d bytes) 000133[0] 1767839923.149978 mapwrite 0x7f8000 thru 0x80708d (0xf08e bytes) 000134[1] 1767839923.285267 read 0x4e8000 thru 0x4eeec1 (0x6ec2 bytes) 000135[0] 1767839923.351760 trunc from 0x95082f to 0x80fb1c (0x140d12 bytes) 000138[0] 1767839923.410737 read 0x224000 thru 0x22bcfb (0x7cfc bytes) 000139[1] 1767839923.472402 mapwrite 0x36f000 thru 0x378508 (0x9509 bytes) 000140[0] 1767839923.584931 trunc from 0x80fb1d to 0x65139b (0x1be781 bytes) 000143[1] 1767839923.655038 trunc from 0x65139c to 0x1e1e59 (0x46f542 bytes) 000149[0] 1767839923.722464 write 0x021000 thru 0x02cc8a (0xbc8b bytes) 000151[0] 1767839923.755761 read 0x038000 thru 0x041a17 (0x9a18 bytes) 000156[0] 1767839923.820833 trunc from 0x1e1e5a to 0x355151 (0x1732f8 bytes) 000158[0] 1767839923.853880 mapread 0x037000 thru 0x04394a (0xc94b bytes) 000161[0] 1767839923.903057 trunc from 0x355152 to 0x458120 (0x102fcf bytes) 000162[0] 1767839923.925487 mapwrite 0x625000 thru 0x633652 (0xe653 bytes) 000164[1] 1767839924.131721 mapread 0x562000 thru 0x56256c (0x56d bytes) 000167[1] 1767839924.178705 mapread 0x5c4000 thru 0x5cf026 (0xb027 bytes) 000170[1] 1767839924.228035 read 0x009000 thru 0x009ad6 (0xad7 bytes) 000171[0] 1767839924.290031 mapwrite 0x1b8000 thru 0x1c0e7f (0x8e80 bytes) 000172[1] 1767839924.367743 write 0x311000 thru 0x311689 (0x68a bytes) 000173[1] 1767839924.378680 mapwrite 0x9ab000 thru 0x9b6c11 (0xbc12 bytes) 000174[1] 1767839924.497253 mapread 0x6d4000 thru 0x6dff70 (0xbf71 bytes) 000175[1] 1767839924.499709 trunc from 0x9b6c12 to 0x6df36e (0x2d78a3 bytes) 000176[0] 1767839924.515228 mapwrite 0x6e2000 thru 0x6f06d9 (0xe6da bytes) 000177[0] 1767839924.685943 write 0x939000 thru 0x9408ed (0x78ee bytes) 000178[0] 1767839924.687382 read 0x6f5000 thru 0x6f94f0 (0x44f1 bytes) 000180[1] 1767839924.733944 trunc from 0x9408ee to 0x0e24bd (0x85e430 bytes) 000181[1] 1767839924.792543 mapread 0x01f000 thru 0x0229ff (0x3a00 bytes) 000183[1] 1767839924.839029 mapwrite 0x7c3000 thru 0x7c3689 (0x68a bytes) 000185[0] 1767839924.947252 read 0x6c7000 thru 0x6cf9d4 (0x89d5 bytes) 000186[0] 1767839924.969792 trunc from 0x7c368a to 0x5f9dd1 (0x1c98b8 bytes) 000188[0] 1767839924.988703 write 0x6d3000 thru 0x6e0335 (0xd336 bytes) 000189[0] 1767839925.007413 mapwrite 0x922000 thru 0x922bfe (0xbff bytes) 000190[0] 1767839925.088363 read 0x51e000 thru 0x521a97 (0x3a98 bytes) 000193[1] 1767839925.151378 read 0x788000 thru 0x7961d1 (0xe1d2 bytes) 000195[1] 1767839925.193871 mapread 0x6c2000 thru 0x6c7bba (0x5bbb bytes) 000197[0] 1767839925.226105 read 0x38b000 thru 0x396ecb (0xbecc bytes) 000201[0] 1767839925.296517 mapwrite 0x211000 thru 0x21cd9d (0xbd9e bytes) 000207[1] 1767839925.437518 write 0x72e000 thru 0x73d711 (0xf712 bytes) 000210[1] 1767839925.448906 mapread 0x311000 thru 0x311543 (0x544 bytes) 000211[1] 1767839925.463797 read 0x823000 thru 0x82f740 (0xc741 bytes) 000212[1] 1767839925.489960 mapwrite 0x447000 thru 0x455ae9 (0xeaea bytes) 000215[0] 1767839925.574831 mapread 0x474000 thru 0x47b70d (0x770e bytes) 000216[0] 1767839925.619285 read 0x715000 thru 0x715300 (0x301 bytes) 000217[0] 1767839925.646224 mapwrite 0x6e3000 thru 0x6eba7f (0x8a80 bytes) 000220[1] 1767839925.758693 read 0x35a000 thru 0x361c12 (0x7c13 bytes) 000221[1] 1767839925.841995 trunc from 0x922bff to 0x548954 (0x3da2aa bytes) 000222[0] 1767839925.872648 write 0x53d000 thru 0x53d0be (0xbf bytes) 000223[1] 1767839925.884933 read 0x17f000 thru 0x18b4aa (0xc4ab bytes) 000225[1] 1767839925.951986 mapwrite 0x0b7000 thru 0x0c38d1 (0xc8d2 bytes) 000226[0] 1767839926.117467 trunc from 0x548955 to 0x9448b8 (0x3fbf64 bytes) 000227[1] 1767839926.155773 trunc from 0x9448b9 to 0x48f31c (0x4b559c bytes) 000228[1] 1767839926.176891 mapwrite 0x362000 thru 0x36901f (0x7020 bytes) 000230[1] 1767839926.251103 mapwrite 0x998000 thru 0x9a43e9 (0xc3ea bytes) 000231[0] 1767839926.338290 write 0x829000 thru 0x832faa (0x9fab bytes) 000232[0] 1767839926.369943 trunc from 0x9a43ea to 0x5791f6 (0x42b1f3 bytes) 000233[0] 1767839926.403476 write 0x421000 thru 0x42581d (0x481e bytes) 000235[0] 1767839926.421233 mapread 0x112000 thru 0x121c7b (0xfc7c bytes) 000237[1] 1767839926.497354 read 0x499000 thru 0x4a4678 (0xb679 bytes) 000238[1] 1767839926.528350 read 0x2c7000 thru 0x2cf59e (0x859f bytes) 000242[0] 1767839926.553656 mapread 0x2d2000 thru 0x2d28bd (0x8be bytes) 000243[1] 1767839926.556456 read 0x396000 thru 0x3a4564 (0xe565 bytes) 000245[1] 1767839926.556978 mapread 0x33a000 thru 0x34072c (0x672d bytes) 000246[0] 1767839926.557399 write 0x12b000 thru 0x13a804 (0xf805 bytes) 000247[1] 1767839926.586930 mapwrite 0x4ee000 thru 0x4f8638 (0xa639 bytes) 000249[0] 1767839926.701066 trunc from 0x5791f7 to 0x150a3c (0x4287ba bytes) 000254[0] 1767839926.762720 mapwrite 0x55a000 thru 0x56580e (0xb80f bytes) 000255[0] 1767839926.881777 trunc from 0x56580f to 0x8f318b (0x38d97d bytes) 000257[1] 1767839926.916095 mapwrite 0x31b000 thru 0x328101 (0xd102 bytes) 000259[0] 1767839927.041522 mapread 0x49a000 thru 0x49c5f2 (0x25f3 bytes) 000260[0] 1767839927.056234 mapwrite 0x165000 thru 0x16a9d0 (0x59d1 bytes) 000261[1] 1767839927.112882 read 0x125000 thru 0x1296de (0x46df bytes) 000262[1] 1767839927.199988 read 0x3ac000 thru 0x3b3e67 (0x7e68 bytes) 000264[1] 1767839927.200482 trunc from 0x8f318c to 0x09eef3 (0x854298 bytes) 000267[0] 1767839927.227019 trunc from 0x09eef4 to 0x2cdcc1 (0x22edce bytes) 000269[1] 1767839927.249884 read 0x0cf000 thru 0x0dcd1c (0xdd1d bytes) 000270[0] 1767839927.274718 mapwrite 0x64c000 thru 0x64f3a6 (0x33a7 bytes) 000273[0] 1767839927.359944 read 0x596000 thru 0x597901 (0x1902 bytes) 000278[1] 1767839927.399906 write 0x265000 thru 0x26e287 (0x9288 bytes) 000279[0] 1767839927.462439 write 0x9d0000 thru 0x9d5bbd (0x5bbe bytes) 000280[1] 1767839927.493499 mapwrite 0x610000 thru 0x612c7f (0x2c80 bytes) 000286[0] 1767839927.616479 mapwrite 0x962000 thru 0x968807 (0x6808 bytes) 000289[0] 1767839927.730322 mapread 0x4fe000 thru 0x502ea0 (0x4ea1 bytes) 000290[0] 1767839927.743260 write 0x9cc000 thru 0x9cfcb2 (0x3cb3 bytes) 000291[0] 1767839927.750041 read 0x971000 thru 0x9775a5 (0x65a6 bytes) 000292[0] 1767839927.755821 read 0x0c2000 thru 0x0c537d (0x337e bytes) 000295[1] 1767839927.817753 mapwrite 0x754000 thru 0x757959 (0x395a bytes) 000296[0] 1767839927.868650 mapread 0x4c5000 thru 0x4c75dd (0x25de bytes) 000297[1] 1767839927.891895 mapwrite 0x6b8000 thru 0x6be1d3 (0x61d4 bytes) 000298[1] 1767839927.967399 write 0x717000 thru 0x71e6e4 (0x76e5 bytes) 000299[1] 1767839927.971023 read 0x321000 thru 0x3244d8 (0x34d9 bytes) 000300[1] 1767839928.065223 mapread 0x554000 thru 0x55755f (0x3560 bytes) 000303[1] 1767839928.080670 mapwrite 0x74e000 thru 0x74e4d3 (0x4d4 bytes) 000304[1] 1767839928.128777 read 0x594000 thru 0x5a1189 (0xd18a bytes) 000305[1] 1767839928.142291 trunc from 0x9d5bbe to 0x2200e7 (0x7b5ad6 bytes) 000306[0] 1767839928.168741 write 0x3ec000 thru 0x3f638c (0xa38d bytes) 000308[1] 1767839928.191208 mapread 0x355000 thru 0x35d213 (0x8214 bytes) 000310[1] 1767839928.252856 mapread 0x36e000 thru 0x36f2bc (0x12bd bytes) 000314[0] 1767839928.279909 trunc from 0x3f638d to 0x092229 (0x364163 bytes) 000315[1] 1767839928.305255 mapread 0x070000 thru 0x07fb59 (0xfb5a bytes) 000316[0] 1767839928.434593 read 0x03c000 thru 0x03f54b (0x354c bytes) 000317[1] 1767839928.447504 trunc from 0x09222a to 0x44fe13 (0x3bdbea bytes) 000318[1] 1767839928.466785 mapread 0x11d000 thru 0x12aa48 (0xda49 bytes) 000319[1] 1767839928.504623 read 0x011000 thru 0x015f55 (0x4f56 bytes) 000320[1] 1767839928.549259 write 0x612000 thru 0x61ec40 (0xcc41 bytes) 000326[1] 1767839928.609041 mapwrite 0x9a8000 thru 0x9b0bc7 (0x8bc8 bytes) 000327[1] 1767839928.733298 write 0x736000 thru 0x7446ec (0xe6ed bytes) 000330[1] 1767839928.764457 mapwrite 0x456000 thru 0x465f38 (0xff39 bytes) 000331[1] 1767839928.823095 trunc from 0x9b0bc8 to 0x709bda (0x2a6fed bytes) 000332[0] 1767839928.845343 mapread 0x672000 thru 0x67d13c (0xb13d bytes) 000333[0] 1767839928.926554 mapwrite 0x7f6000 thru 0x8036d0 (0xd6d1 bytes) 000335[0] 1767839929.087811 trunc from 0x8036d1 to 0x405dfc (0x3fd8d4 bytes) 000336[0] 1767839929.132413 trunc from 0x405dfd to 0x341c7c (0xc4180 bytes) 000340[1] 1767839929.177504 mapread 0x070000 thru 0x077682 (0x7683 bytes) 000342[1] 1767839929.222334 write 0x459000 thru 0x45f7df (0x67e0 bytes) 000343[1] 1767839929.228783 write 0x16b000 thru 0x175ca6 (0xaca7 bytes) 000344[0] 1767839929.242617 mapread 0x04e000 thru 0x059ad0 (0xbad1 bytes) 000345[0] 1767839929.344390 mapread 0x1e5000 thru 0x1f2499 (0xd49a bytes) 000346[1] 1767839929.400880 mapread 0x017000 thru 0x01b87d (0x487e bytes) 000347[0] 1767839929.424524 read 0x29f000 thru 0x2a16af (0x26b0 bytes) 000348[0] 1767839929.456417 write 0x77a000 thru 0x77a085 (0x86 bytes) 000353[1] 1767839929.515472 mapwrite 0x94d000 thru 0x94f924 (0x2925 bytes) 000354[0] 1767839929.629791 read 0x4d5000 thru 0x4de546 (0x9547 bytes) 000356[0] 1767839929.711736 write 0x2c0000 thru 0x2c0346 (0x347 bytes) 000357[0] 1767839929.728899 mapwrite 0x04c000 thru 0x04e280 (0x2281 bytes) 000358[0] 1767839929.789937 read 0x674000 thru 0x67768f (0x3690 bytes) 000360[0] 1767839929.859575 mapwrite 0x055000 thru 0x061ba5 (0xcba6 bytes) 000361[1] 1767839929.970295 trunc from 0x94f925 to 0x6b80d1 (0x297853 bytes) 000362[0] 1767839930.010320 mapwrite 0x2f7000 thru 0x3047ff (0xd800 bytes) 000363[1] 1767839930.163396 mapread 0x582000 thru 0x58e563 (0xc564 bytes) 000364[1] 1767839930.223208 trunc from 0x6b80d2 to 0x129465 (0x58ec6c bytes) 000365[1] 1767839930.269635 read 0x06f000 thru 0x0725bd (0x35be bytes) 000366[1] 1767839930.287614 read 0x0fc000 thru 0x0fe221 (0x2222 bytes) 000369[1] 1767839930.334223 read 0x027000 thru 0x02cb70 (0x5b71 bytes) 000370[1] 1767839930.334589 read 0x07c000 thru 0x08a2a8 (0xe2a9 bytes) 000371[0] 1767839930.334800 mapread 0x0da000 thru 0x0e67e9 (0xc7ea bytes) 000375[1] 1767839930.401784 write 0x187000 thru 0x194ed4 (0xded5 bytes) 000376[1] 1767839930.423767 write 0x6e2000 thru 0x6e9231 (0x7232 bytes) 000377[0] 1767839930.440004 write 0x251000 thru 0x25a76a (0x976b bytes) 000379[1] 1767839930.468631 trunc from 0x6e9232 to 0x4c5a44 (0x2237ed bytes) 000380[1] 1767839930.493567 mapread 0x1e7000 thru 0x1ebbbb (0x4bbc bytes) 000381[0] 1767839930.563042 mapread 0x0d7000 thru 0x0e32e9 (0xc2ea bytes) 000383[0] 1767839930.650604 mapread 0x430000 thru 0x43c413 (0xc414 bytes) 000386[0] 1767839930.697845 trunc from 0x4c5a45 to 0x61d64c (0x157c08 bytes) 000387[0] 1767839930.726719 trunc from 0x61d64d to 0x4446d9 (0x1d8f73 bytes) 000388[1] 1767839930.734778 write 0x40d000 thru 0x410e1b (0x3e1c bytes) 000389[0] 1767839930.745216 mapwrite 0x6da000 thru 0x6e3c4c (0x9c4d bytes) 000392[0] 1767839930.857308 mapread 0x237000 thru 0x2405cf (0x95d0 bytes) 000393[1] 1767839930.880499 trunc from 0x6e3c4d to 0x011ff9 (0x6d1c53 bytes) 000394[0] 1767839930.907009 trunc from 0x011ffa to 0x8aa42c (0x898433 bytes) 000395[1] 1767839930.934049 trunc from 0x8aa42d to 0x3fdebe (0x4ac56e bytes) 000396[1] 1767839930.946521 read 0x293000 thru 0x293ac3 (0xac4 bytes) 000397[0] 1767839930.963681 read 0x0b0000 thru 0x0b835c (0x835d bytes) 000400[1] 1767839931.018521 write 0x7dd000 thru 0x7e97e6 (0xc7e7 bytes) 000401[0] 1767839931.049019 mapread 0x449000 thru 0x449514 (0x515 bytes) 000402[1] 1767839931.121604 mapwrite 0x984000 thru 0x991c25 (0xdc26 bytes) 000403[0] 1767839931.248918 trunc from 0x991c26 to 0x26eb75 (0x7230b0 bytes) 000404[0] 1767839931.274987 read 0x05e000 thru 0x06c46f (0xe470 bytes) 000405[1] 1767839931.350900 read 0x0e4000 thru 0x0e829f (0x42a0 bytes) 000407[0] 1767839931.389991 write 0x242000 thru 0x25015e (0xe15f bytes) truncating to largest ever: 0x9e12f0 000414[0] 1767839931.473797 trunc from 0x26eb76 to 0x9e12ef (0x77277a bytes) 000416[0] 1767839931.500542 read 0x25d000 thru 0x266ce6 (0x9ce7 bytes) 000417[1] 1767839931.567622 trunc from 0x9e12f0 to 0x5f548e (0x3ebe61 bytes) 000418[1] 1767839931.634965 mapread 0x375000 thru 0x37e4be (0x94bf bytes) 000420[0] 1767839931.695032 trunc from 0x5f548f to 0x0c4339 (0x531155 bytes) 000424[0] 1767839931.768388 mapread 0x068000 thru 0x06a297 (0x2298 bytes) 000426[1] 1767839931.793837 trunc from 0x0c433a to 0x794999 (0x6d0660 bytes) 000428[0] 1767839931.829366 read 0x431000 thru 0x433f05 (0x2f06 bytes) 000429[0] 1767839931.920198 mapread 0x39e000 thru 0x3a7a5d (0x9a5e bytes) 000430[1] 1767839931.985742 write 0x2b7000 thru 0x2b921b (0x221c bytes) 000431[0] 1767839932.039217 read 0x6a1000 thru 0x6a3301 (0x2302 bytes) 000433[0] 1767839932.083993 mapread 0x662000 thru 0x66a920 (0x8921 bytes) 000434[1] 1767839932.121079 trunc from 0x79499a to 0x67b819 (0x119180 bytes) 000436[1] 1767839932.173420 mapread 0x370000 thru 0x379888 (0x9889 bytes) 000437[1] 1767839932.233221 mapwrite 0x0a3000 thru 0x0a6259 (0x325a bytes) 000438[1] 1767839932.309377 trunc from 0x67b81a to 0x64eb02 (0x2cd17 bytes) 000439[1] 1767839932.339619 write 0x6d2000 thru 0x6d6fde (0x4fdf bytes) 000442[1] 1767839932.378719 trunc from 0x6d6fdf to 0x05d250 (0x679d8e bytes) 000444[1] 1767839932.400380 mapread 0x035000 thru 0x038176 (0x3177 bytes) 000446[0] 1767839932.433875 mapwrite 0x6d5000 thru 0x6e03ab (0xb3ac bytes) 000451[1] 1767839932.633122 write 0x1ea000 thru 0x1ef4d4 (0x54d5 bytes) 000454[1] 1767839932.673218 read 0x590000 thru 0x5945df (0x45e0 bytes) 000457[1] 1767839932.729765 mapread 0x1bd000 thru 0x1c70e9 (0xa0ea bytes) 000459[0] 1767839932.792591 mapwrite 0x0d8000 thru 0x0d9e28 (0x1e29 bytes) 000460[0] 1767839932.873584 mapread 0x6d5000 thru 0x6dfb22 (0xab23 bytes) 000461[0] 1767839932.924446 mapread 0x599000 thru 0x5a7cbf (0xecc0 bytes) 000463[1] 1767839933.003671 mapwrite 0x260000 thru 0x26b153 (0xb154 bytes) 000464[1] 1767839933.097102 write 0x496000 thru 0x4a2095 (0xc096 bytes) 000466[1] 1767839933.115766 trunc from 0x6e03ac to 0x2fd405 (0x3e2fa6 bytes) 000467[0] 1767839933.128454 trunc from 0x2fd406 to 0x7fefa7 (0x501ba2 bytes) 000469[0] 1767839933.176432 mapwrite 0x847000 thru 0x8475c8 (0x5c9 bytes) 000471[1] 1767839933.264260 write 0x75a000 thru 0x7605d7 (0x65d8 bytes) 000476[1] 1767839933.360898 mapwrite 0x934000 thru 0x93b022 (0x7023 bytes) truncating to largest ever: 0x9ee82d 000477[1] 1767839933.473741 trunc from 0x93b023 to 0x9ee82c (0xb380a bytes) 000482[1] 1767839933.511012 write 0x9dc000 thru 0x9e6147 (0xa148 bytes) 000483[0] 1767839933.514014 trunc from 0x9ee82d to 0x729aa5 (0x2c4d87 bytes) 000485[1] 1767839933.558115 mapwrite 0x2aa000 thru 0x2ae129 (0x412a bytes) 000486[0] 1767839933.638430 write 0x3e0000 thru 0x3eb883 (0xb884 bytes) 000487[0] 1767839933.658211 mapread 0x0fd000 thru 0x10bdb6 (0xedb7 bytes) 000489[1] 1767839933.724847 read 0x38e000 thru 0x39cd9a (0xed9b bytes) 000490[0] 1767839933.781745 read 0x53a000 thru 0x540f7e (0x6f7f bytes) 000493[1] 1767839933.814542 read 0x503000 thru 0x50e134 (0xb135 bytes) 000495[0] 1767839933.815338 write 0x960000 thru 0x969cda (0x9cdb bytes) 000496[0] 1767839933.831666 mapread 0x968000 thru 0x969ad5 (0x1ad6 bytes) 000497[0] 1767839933.831987 read 0x939000 thru 0x93cc7f (0x3c80 bytes) 000498[1] 1767839933.855059 read 0x71e000 thru 0x720ec4 (0x2ec5 bytes) 000499[1] 1767839933.901987 write 0x28e000 thru 0x290983 (0x2984 bytes) 000500[1] 1767839933.933841 write 0x9a9000 thru 0x9ae05c (0x505d bytes) All operations completed A-OK! PASS 16a (37s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 21:38:57 (1767839937) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.176215 s, 11.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.162573 s, 12.9 MB/s Chance of close/open is 1 in 50 Seed set to 9938 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x8a7675 000001[0] 1767839938.841905 trunc from 00000000 to 0x8a7674 (0x8a7675 bytes) 000002[1] 1767839938.876789 read 0x407725 thru 0x40806f (0x94b bytes) 000003[1] 1767839938.887623 read 0x47d733 thru 0x47f41e (0x1cec bytes) 000007[1] 1767839938.927514 trunc from 0x8a7675 to 0x1b4247 (0x6f342d bytes) 000008[1] 1767839938.961964 mapread 0x1952fa thru 0x195e0c (0xb13 bytes) 000009[1] 1767839938.965976 mapwrite 0x97ff5a thru 0x980260 (0x307 bytes) 000012[1] 1767839939.135006 mapwrite 0x30d79a thru 0x30f26a (0x1ad1 bytes) 000014[0] 1767839939.178881 write 0x9ce3e9 thru 0x9ce5b0 (0x1c8 bytes) 000015[1] 1767839939.195925 write 0x0bc4e7 thru 0x0bde53 (0x196d bytes) 000016[1] 1767839939.207747 read 0x89fff3 thru 0x8a0bd5 (0xbe3 bytes) 000019[1] 1767839939.225781 mapread 0x87bc79 thru 0x87c9d3 (0xd5b bytes) 000020[0] 1767839939.235764 mapread 0x271d16 thru 0x2736d7 (0x19c2 bytes) 000021[1] 1767839939.260471 read 0x9b1533 thru 0x9b31c1 (0x1c8f bytes) 000023[1] 1767839939.276124 mapwrite 0x364606 thru 0x364dad (0x7a8 bytes) 000026[1] 1767839939.310528 mapread 0x4069c6 thru 0x408837 (0x1e72 bytes) 000027[0] 1767839939.321057 write 0x9f99c4 thru 0x9fa096 (0x6d3 bytes) 000029[0] 1767839939.344470 mapwrite 0x1a89b3 thru 0x1aa69f (0x1ced bytes) 000031[1] 1767839939.403901 read 0x71a38b thru 0x71b50c (0x1182 bytes) 000032[1] 1767839939.445564 write 0x4931de thru 0x4939bf (0x7e2 bytes) 000033[0] 1767839939.481256 mapwrite 0x7b10e4 thru 0x7b2a85 (0x19a2 bytes) 000035[1] 1767839939.592794 mapwrite 0x7056e5 thru 0x7056f9 (0x15 bytes) 000036[0] 1767839939.667868 mapread 0x057d39 thru 0x058358 (0x620 bytes) 000037[1] 1767839939.709482 mapread 0x641ef4 thru 0x643b77 (0x1c84 bytes) 000038[1] 1767839939.731756 mapwrite 0x650252 thru 0x650ab2 (0x861 bytes) 000039[0] 1767839939.794663 trunc from 0x9fa097 to 0x514b24 (0x4e5572 bytes) 000040[1] 1767839939.847051 read 0x09bd5b thru 0x09c484 (0x72a bytes) 000042[0] 1767839939.874584 mapwrite 0x618049 thru 0x618a79 (0xa31 bytes) 000044[1] 1767839939.979283 write 0x14f1cf thru 0x150082 (0xeb4 bytes) 000045[1] 1767839940.051841 write 0x9b11d8 thru 0x9b1cb8 (0xae1 bytes) 000046[1] 1767839940.077219 mapwrite 0x976f43 thru 0x977772 (0x830 bytes) 000048[0] 1767839940.142436 trunc from 0x9b1cb9 to 0x7368bf (0x27b3f9 bytes) 000049[1] 1767839940.161285 mapwrite 0x97f68d thru 0x9807c6 (0x113a bytes) 000050[0] 1767839940.313666 read 0x971a2b thru 0x9726b1 (0xc87 bytes) 000051[1] 1767839940.332682 trunc from 0x9807c7 to 0x2c69b5 (0x6b9e11 bytes) truncating to largest ever: 0x8af5a8 000053[0] 1767839940.390906 trunc from 0x2c69b6 to 0x8af5a7 (0x5e8bf2 bytes) 000056[1] 1767839940.464932 mapread 0x71cef1 thru 0x71de67 (0xf77 bytes) 000057[1] 1767839940.508697 read 0x5f4b1e thru 0x5f4d90 (0x273 bytes) 000060[0] 1767839940.550705 read 0x0d7d56 thru 0x0d80ed (0x398 bytes) 000061[1] 1767839940.632614 mapwrite 0x3b1194 thru 0x3b1c7a (0xae7 bytes) truncating to largest ever: 0x9f40c7 000063[0] 1767839940.685965 trunc from 0x8af5a8 to 0x9f40c6 (0x144b1f bytes) 000065[0] 1767839940.729358 trunc from 0x9f40c7 to 0x05ebc7 (0x9954ff bytes) 000066[1] 1767839940.768066 mapread 0x05cf67 thru 0x05d3a2 (0x43c bytes) 000068[0] 1767839940.803296 write 0x060eb3 thru 0x06125c (0x3aa bytes) 000070[1] 1767839940.826203 read 0x0182ae thru 0x019d38 (0x1a8b bytes) 000072[0] 1767839940.862675 read 0x018a1d thru 0x019e91 (0x1475 bytes) 000074[0] 1767839940.881421 mapwrite 0x2243b8 thru 0x22510e (0xd57 bytes) 000076[0] 1767839941.061651 mapwrite 0x4b4b4a thru 0x4b4b8d (0x44 bytes) 000077[0] 1767839941.114406 write 0x9e05c8 thru 0x9e1b95 (0x15ce bytes) 000078[0] 1767839941.120752 mapwrite 0x6e3820 thru 0x6e42f6 (0xad7 bytes) 000080[1] 1767839941.205818 mapread 0x5faa98 thru 0x5fb1e6 (0x74f bytes) 000081[0] 1767839941.239111 mapwrite 0x3c25c2 thru 0x3c33c2 (0xe01 bytes) 000082[1] 1767839941.292953 write 0x0387cc thru 0x039e1d (0x1652 bytes) 000091[0] 1767839941.435981 mapwrite 0x48ec36 thru 0x48fcb9 (0x1084 bytes) 000092[1] 1767839941.495106 write 0x7cc2a6 thru 0x7cdc5c (0x19b7 bytes) 000095[1] 1767839941.548208 trunc from 0x9e1b96 to 0x8f5936 (0xec25f bytes) 000097[0] 1767839941.578301 write 0x3d83f5 thru 0x3d8f5d (0xb69 bytes) 000098[1] 1767839941.600006 write 0x6900ea thru 0x69136b (0x1282 bytes) 000099[0] 1767839941.627126 mapwrite 0x4b1f8e thru 0x4b2f83 (0xff6 bytes) 000103[0] 1767839941.713573 write 0x57cc73 thru 0x57e63d (0x19cb bytes) 000104[1] 1767839941.722853 write 0x3a21c3 thru 0x3a246d (0x2ab bytes) 000105[1] 1767839941.776859 read 0x8481c3 thru 0x84966d (0x14ab bytes) 000106[0] 1767839941.822909 mapread 0x6eaf95 thru 0x6eb0c3 (0x12f bytes) 000107[0] 1767839941.845110 trunc from 0x8f5937 to 0x259783 (0x69c1b3 bytes) 000108[1] 1767839941.891868 read 0x05fde4 thru 0x060f77 (0x1194 bytes) 000109[0] 1767839941.935202 read 0x09b8e6 thru 0x09baab (0x1c6 bytes) 000110[1] 1767839941.972307 trunc from 0x259784 to 0x88e8d4 (0x635151 bytes) 000111[0] 1767839942.032709 write 0x5e2eb9 thru 0x5e3373 (0x4bb bytes) 000112[1] 1767839942.066937 write 0x8755be thru 0x87672d (0x1170 bytes) 000115[0] 1767839942.118610 write 0x8c3aa2 thru 0x8c3c9d (0x1fc bytes) 000116[0] 1767839942.163660 mapread 0x8a263f thru 0x8a29e4 (0x3a6 bytes) 000117[0] 1767839942.240470 trunc from 0x8c3c9e to 0x6b38be (0x2103df bytes) 000118[0] 1767839942.261935 mapwrite 0x1e3521 thru 0x1e4e2a (0x190a bytes) 000120[1] 1767839942.307877 mapread 0x658180 thru 0x65820c (0x8d bytes) 000123[1] 1767839942.377555 mapwrite 0x09968c thru 0x09a4f4 (0xe69 bytes) 000125[1] 1767839942.521213 read 0x4c44e6 thru 0x4c6057 (0x1b72 bytes) 000127[1] 1767839942.576175 read 0x5ba372 thru 0x5ba86f (0x4fe bytes) 000128[1] 1767839942.599923 trunc from 0x6b38bf to 0x92bbf4 (0x278336 bytes) 000132[0] 1767839942.659595 write 0x9894a3 thru 0x98a578 (0x10d6 bytes) 000137[0] 1767839942.730232 mapread 0x8e5f72 thru 0x8e6545 (0x5d4 bytes) 000140[1] 1767839942.794048 mapwrite 0x8df426 thru 0x8e05b9 (0x1194 bytes) 000143[0] 1767839942.905272 write 0x539a24 thru 0x53a615 (0xbf2 bytes) 000144[0] 1767839942.922530 mapwrite 0x0722f8 thru 0x073595 (0x129e bytes) 000145[0] 1767839942.982512 mapread 0x0c3dcb thru 0x0c50d4 (0x130a bytes) 000146[0] 1767839942.995931 mapwrite 0x9040c5 thru 0x905601 (0x153d bytes) 000147[1] 1767839943.038913 trunc from 0x98a579 to 0x1042a6 (0x8862d2 bytes) 000148[0] 1767839943.088842 trunc from 0x1042a7 to 0x6e5a77 (0x5e17d1 bytes) 000149[1] 1767839943.121670 write 0x461824 thru 0x461a5c (0x239 bytes) 000150[1] 1767839943.152313 trunc from 0x6e5a78 to 0x5a9602 (0x13c475 bytes) 000151[1] 1767839943.185822 trunc from 0x5a9603 to 0x0decb6 (0x4ca94c bytes) 000152[1] 1767839943.198460 mapwrite 0x09940f thru 0x099d99 (0x98b bytes) 000156[1] 1767839943.337530 read 0x0d7152 thru 0x0d77f7 (0x6a6 bytes) 000160[0] 1767839943.370589 mapwrite 0x2b4b03 thru 0x2b6487 (0x1985 bytes) 000162[1] 1767839943.456860 mapwrite 0x545adf thru 0x545f76 (0x498 bytes) 000163[0] 1767839943.591941 write 0x637b9f thru 0x639582 (0x19e4 bytes) 000164[0] 1767839943.617320 mapwrite 0x866642 thru 0x867565 (0xf24 bytes) 000165[0] 1767839943.700094 mapwrite 0x0ed571 thru 0x0ee4d2 (0xf62 bytes) 000168[1] 1767839943.793889 trunc from 0x867566 to 0x486ddc (0x3e0789 bytes) 000171[1] 1767839943.845658 write 0x103b69 thru 0x104517 (0x9af bytes) 000172[0] 1767839943.876915 read 0x113ce6 thru 0x1157f7 (0x1b12 bytes) 000173[0] 1767839943.909102 mapwrite 0x61f703 thru 0x620c5d (0x155b bytes) 000174[0] 1767839944.046980 read 0x4f0c9b thru 0x4f29a1 (0x1d07 bytes) 000177[0] 1767839944.088135 mapread 0x38effa thru 0x38fe6b (0xe72 bytes) 000178[0] 1767839944.092565 mapwrite 0x3ddd61 thru 0x3de1cb (0x46b bytes) 000179[1] 1767839944.158210 mapread 0x495a49 thru 0x496fc2 (0x157a bytes) 000180[0] 1767839944.187046 trunc from 0x620c5e to 0x77c759 (0x15bafc bytes) 000181[0] 1767839944.239310 mapread 0x775e3e thru 0x777154 (0x1317 bytes) 000182[0] 1767839944.268591 trunc from 0x77c75a to 0x5af7ec (0x1ccf6d bytes) 000186[0] 1767839944.300883 read 0x03c956 thru 0x03ce15 (0x4c0 bytes) 000187[1] 1767839944.337619 mapwrite 0x6729c9 thru 0x6730a2 (0x6da bytes) 000191[0] 1767839944.480215 trunc from 0x6730a3 to 0x775315 (0x102273 bytes) 000192[1] 1767839944.516333 mapread 0x0cde75 thru 0x0ce5df (0x76b bytes) 000194[0] 1767839944.569654 mapwrite 0x4999af thru 0x49a45b (0xaad bytes) 000195[0] 1767839944.645709 trunc from 0x775316 to 0x75444d (0x20ec8 bytes) 000196[0] 1767839944.661809 trunc from 0x75444e to 0x641302 (0x11314b bytes) 000199[1] 1767839944.678867 write 0x57e4c0 thru 0x57fe8e (0x19cf bytes) 000200[1] 1767839944.720095 write 0x3c7bac thru 0x3c7c0d (0x62 bytes) 000201[1] 1767839944.742472 mapwrite 0x50b1af thru 0x50c0be (0xf10 bytes) 000202[0] 1767839944.794523 trunc from 0x641303 to 0x230f45 (0x4103bd bytes) 000204[0] 1767839944.836219 mapwrite 0x925aab thru 0x92687f (0xdd5 bytes) 000206[1] 1767839944.949206 trunc from 0x926880 to 0x3eb0c9 (0x53b7b6 bytes) 000207[1] 1767839945.009869 trunc from 0x3eb0ca to 0x25c191 (0x18ef38 bytes) 000208[1] 1767839945.024729 write 0x17b6dc thru 0x17c2f4 (0xc19 bytes) 000209[1] 1767839945.051840 write 0x50fd1e thru 0x511699 (0x197c bytes) 000211[0] 1767839945.053526 trunc from 0x51169a to 0x1da46f (0x33722a bytes) 000212[0] 1767839945.097626 write 0x2c745d thru 0x2c7cc2 (0x866 bytes) 000214[0] 1767839945.133908 trunc from 0x2c7cc3 to 0x8603c8 (0x598706 bytes) 000216[0] 1767839945.190553 read 0x0c8eee thru 0x0c97a0 (0x8b3 bytes) 000218[0] 1767839945.310237 read 0x85d18b thru 0x85f161 (0x1fd7 bytes) 000220[0] 1767839945.336077 trunc from 0x8603c9 to 0x74c4ee (0x113eda bytes) 000221[1] 1767839945.354342 mapwrite 0x28433e thru 0x284654 (0x317 bytes) 000222[0] 1767839945.438043 mapread 0x36dad4 thru 0x36f72c (0x1c59 bytes) 000225[0] 1767839945.471064 write 0x0d1507 thru 0x0d1857 (0x351 bytes) 000226[1] 1767839945.481623 write 0x6a9a61 thru 0x6a9e06 (0x3a6 bytes) 000228[0] 1767839945.515650 trunc from 0x74c4ef to 0x453810 (0x2f8cde bytes) 000231[1] 1767839945.560071 read 0x1a1de6 thru 0x1a245d (0x678 bytes) 000232[0] 1767839945.595375 write 0x8464d5 thru 0x847327 (0xe53 bytes) 000233[0] 1767839945.605001 read 0x1d60bf thru 0x1d6c76 (0xbb8 bytes) 000238[0] 1767839945.688953 mapwrite 0x55eb76 thru 0x55fbb7 (0x1042 bytes) 000239[0] 1767839945.765435 trunc from 0x847328 to 0x3714af (0x4d5e78 bytes) 000241[1] 1767839945.797093 write 0x1afc9c thru 0x1b19a4 (0x1d09 bytes) 000242[0] 1767839945.818966 mapread 0x0a3ce3 thru 0x0a571b (0x1a39 bytes) 000244[1] 1767839945.866587 trunc from 0x3714b0 to 0x58a08b (0x218bdc bytes) 000247[1] 1767839945.918245 write 0x85f93b thru 0x860ee4 (0x15aa bytes) 000248[1] 1767839945.919555 mapwrite 0x2a3ded thru 0x2a4ae8 (0xcfc bytes) 000251[1] 1767839946.002636 write 0x9b9856 thru 0x9baa28 (0x11d3 bytes) 000258[1] 1767839946.032961 read 0x4ad980 thru 0x4ade4a (0x4cb bytes) 000259[0] 1767839946.033306 trunc from 0x9baa29 to 0x23d292 (0x77d796 bytes) 000261[1] 1767839946.118743 trunc from 0x23d293 to 0x540cc3 (0x303a31 bytes) 000263[1] 1767839946.167777 write 0x727067 thru 0x727c6d (0xc07 bytes) 000264[1] 1767839946.171168 mapread 0x4692b5 thru 0x469dc3 (0xb0f bytes) 000269[1] 1767839946.198094 trunc from 0x727c6e to 0x2bca36 (0x46b237 bytes) 000270[1] 1767839946.241236 read 0x019f7c thru 0x01beb6 (0x1f3b bytes) 000271[0] 1767839946.296308 mapread 0x29f2d4 thru 0x29ffc9 (0xcf6 bytes) 000280[0] 1767839946.371287 read 0x050760 thru 0x050ecc (0x76d bytes) 000282[0] 1767839946.401961 write 0x27eadb thru 0x27ee6f (0x395 bytes) 000283[0] 1767839946.425266 mapwrite 0x21101d thru 0x2128b9 (0x189d bytes) 000284[1] 1767839946.489427 write 0x57578d thru 0x576683 (0xef7 bytes) 000288[0] 1767839946.547437 mapread 0x2bb518 thru 0x2bd0b2 (0x1b9b bytes) 000289[1] 1767839946.588250 trunc from 0x576684 to 0x6b47a0 (0x13e11d bytes) 000290[0] 1767839946.607071 read 0x3bc91f thru 0x3bdf69 (0x164b bytes) 000291[0] 1767839946.629921 write 0x486e40 thru 0x4884bd (0x167e bytes) 000292[1] 1767839946.677189 read 0x0097fd thru 0x00b76d (0x1f71 bytes) 000293[0] 1767839946.751255 mapwrite 0x67dc1b thru 0x67e292 (0x678 bytes) 000294[1] 1767839946.788439 read 0x38bd03 thru 0x38dbc4 (0x1ec2 bytes) 000295[0] 1767839946.794505 write 0x86390c thru 0x865875 (0x1f6a bytes) 000297[0] 1767839946.795766 read 0x5b895b thru 0x5b9a84 (0x112a bytes) 000300[1] 1767839946.796384 write 0x1192b2 thru 0x1195ce (0x31d bytes) 000301[0] 1767839946.827678 write 0x17acca thru 0x17b378 (0x6af bytes) 000302[1] 1767839946.847414 trunc from 0x865876 to 0x5da5ab (0x28b2ca bytes) 000303[0] 1767839946.876281 mapread 0x2f4ddf thru 0x2f60b0 (0x12d2 bytes) 000305[1] 1767839946.912727 read 0x58b5fb thru 0x58d011 (0x1a17 bytes) 000307[0] 1767839946.933166 read 0x10c98f thru 0x10e8a9 (0x1f1b bytes) 000308[0] 1767839946.992357 trunc from 0x5da5ac to 0x37c327 (0x25e284 bytes) 000309[0] 1767839947.040346 mapwrite 0x795b99 thru 0x79658d (0x9f5 bytes) 000312[1] 1767839947.150825 mapwrite 0x572944 thru 0x574904 (0x1fc1 bytes) 000313[0] 1767839947.219529 write 0x0be7fb thru 0x0bf0b7 (0x8bd bytes) 000314[1] 1767839947.238575 write 0x4fedab thru 0x4ff67c (0x8d2 bytes) 000317[1] 1767839947.299461 mapread 0x719985 thru 0x71a34e (0x9ca bytes) 000319[0] 1767839947.323745 write 0x878cfc thru 0x878f0e (0x213 bytes) 000322[1] 1767839947.361330 mapread 0x80191d thru 0x802294 (0x978 bytes) 000323[0] 1767839947.385024 trunc from 0x878f0f to 0x6166eb (0x262823 bytes) 000324[0] 1767839947.408102 write 0x9bd328 thru 0x9bec68 (0x1941 bytes) 000325[0] 1767839947.430788 trunc from 0x9bec69 to 0x3fb426 (0x5c3842 bytes) 000328[0] 1767839947.480850 trunc from 0x3fb427 to 0x924d8f (0x529969 bytes) 000329[0] 1767839947.501809 mapread 0x135f9a thru 0x136890 (0x8f7 bytes) 000333[1] 1767839947.530552 trunc from 0x924d90 to 0x504291 (0x420afe bytes) 000335[1] 1767839947.573936 mapwrite 0x81c28f thru 0x81dd94 (0x1b06 bytes) 000336[1] 1767839947.637861 mapwrite 0x833244 thru 0x833fe8 (0xda5 bytes) 000337[0] 1767839947.700510 trunc from 0x833fe9 to 0x5d9343 (0x25aca5 bytes) 000338[1] 1767839947.747492 trunc from 0x5d9344 to 0x41ce20 (0x1bc523 bytes) 000340[1] 1767839947.810083 trunc from 0x41ce21 to 0x2bb61f (0x161801 bytes) 000342[1] 1767839947.831621 read 0x122e26 thru 0x122e6f (0x4a bytes) 000343[0] 1767839947.850405 trunc from 0x2bb620 to 0x63af9c (0x37f97d bytes) 000344[0] 1767839947.892019 trunc from 0x63af9d to 0x0d3ad2 (0x5674ca bytes) 000345[0] 1767839947.898828 mapread 0x008387 thru 0x0093d7 (0x1051 bytes) 000346[1] 1767839947.916086 trunc from 0x0d3ad3 to 0x5eed9a (0x51b2c8 bytes) 000347[0] 1767839947.980627 write 0x715e36 thru 0x7161fd (0x3c8 bytes) 000350[0] 1767839948.043049 trunc from 0x7161fe to 0x055fa8 (0x6c0255 bytes) 000354[0] 1767839948.109757 trunc from 0x055fa9 to 0x7f5f9a (0x79fff2 bytes) 000355[0] 1767839948.151780 mapwrite 0x2be670 thru 0x2c042c (0x1dbd bytes) 000360[1] 1767839948.372263 mapread 0x1b4847 thru 0x1b50fc (0x8b6 bytes) 000361[0] 1767839948.413482 read 0x467029 thru 0x467528 (0x500 bytes) 000362[1] 1767839948.443554 write 0x9e7455 thru 0x9e7f7d (0xb29 bytes) 000365[1] 1767839948.501801 write 0x966358 thru 0x9664d0 (0x179 bytes) 000367[1] 1767839948.540928 mapwrite 0x16cd3c thru 0x16dcea (0xfaf bytes) 000369[0] 1767839948.636240 write 0x00ac52 thru 0x00aec1 (0x270 bytes) 000371[1] 1767839948.659912 write 0x892493 thru 0x89280b (0x379 bytes) 000375[1] 1767839948.740798 read 0x4755ea thru 0x476d1d (0x1734 bytes) 000377[1] 1767839948.759795 write 0x37e38e thru 0x37f0d9 (0xd4c bytes) 000379[0] 1767839948.766983 mapread 0x768556 thru 0x768ad9 (0x584 bytes) 000382[0] 1767839948.831618 mapread 0x9d0021 thru 0x9d1412 (0x13f2 bytes) 000383[1] 1767839948.857606 mapread 0x839c66 thru 0x83a500 (0x89b bytes) 000384[0] 1767839948.866848 trunc from 0x9e7f7e to 0x16c835 (0x87b748 bytes) 000387[0] 1767839948.947563 mapread 0x014baa thru 0x01524b (0x6a2 bytes) 000389[0] 1767839948.973660 mapwrite 0x87c3e9 thru 0x87c9c4 (0x5dc bytes) 000393[1] 1767839949.109726 mapwrite 0x2ec30b thru 0x2ed91c (0x1612 bytes) 000394[1] 1767839949.191614 write 0x4ea069 thru 0x4ebec1 (0x1e59 bytes) 000397[1] 1767839949.251508 read 0x7b620f thru 0x7b6263 (0x55 bytes) 000400[0] 1767839949.297376 write 0x73a90c thru 0x73b73c (0xe31 bytes) 000402[0] 1767839949.329609 write 0x2f2b4b thru 0x2f2e92 (0x348 bytes) 000403[1] 1767839949.342817 mapread 0x6b1c14 thru 0x6b3554 (0x1941 bytes) 000405[1] 1767839949.389735 mapread 0x1d2b90 thru 0x1d393c (0xdad bytes) 000406[0] 1767839949.407335 read 0x5583df thru 0x55979f (0x13c1 bytes) 000408[1] 1767839949.421113 mapwrite 0x6d76c8 thru 0x6d8076 (0x9af bytes) 000410[1] 1767839949.472758 write 0x5065f5 thru 0x508496 (0x1ea2 bytes) 000411[0] 1767839949.486208 mapread 0x2839f5 thru 0x284af1 (0x10fd bytes) 000412[0] 1767839949.518583 read 0x20bd96 thru 0x20c605 (0x870 bytes) 000414[0] 1767839949.556555 trunc from 0x87c9c5 to 0x8d4e1f (0x5845b bytes) 000415[0] 1767839949.621784 read 0x47b611 thru 0x47b7ae (0x19e bytes) 000417[0] 1767839949.682740 read 0x094b1f thru 0x094e3b (0x31d bytes) 000418[1] 1767839949.718764 trunc from 0x8d4e20 to 0x88cc7e (0x481a1 bytes) 000419[0] 1767839949.766116 mapread 0x2b24a9 thru 0x2b40ce (0x1c26 bytes) 000420[0] 1767839949.804177 read 0x84f14a thru 0x85004b (0xf02 bytes) 000421[1] 1767839949.819051 write 0x2cc1a2 thru 0x2ccc5f (0xabe bytes) 000423[1] 1767839949.831870 mapwrite 0x7c0708 thru 0x7c21a8 (0x1aa1 bytes) 000424[0] 1767839949.880391 mapwrite 0x475599 thru 0x47613c (0xba4 bytes) 000425[0] 1767839949.966992 write 0x1239cf thru 0x1248e5 (0xf17 bytes) 000426[0] 1767839949.992875 mapwrite 0x5fa202 thru 0x5fbf2b (0x1d2a bytes) 000428[0] 1767839950.065910 mapread 0x7b80a1 thru 0x7b8b76 (0xad6 bytes) 000431[0] 1767839950.111626 read 0x2df3a1 thru 0x2e01c0 (0xe20 bytes) 000432[0] 1767839950.162632 trunc from 0x88cc7f to 0x655fcb (0x236cb3 bytes) 000435[0] 1767839950.192584 read 0x41d10e thru 0x41d272 (0x165 bytes) 000437[0] 1767839950.208477 write 0x585fd0 thru 0x5866a4 (0x6d5 bytes) 000439[0] 1767839950.218161 mapread 0x2d4d3a thru 0x2d5aaa (0xd71 bytes) 000440[0] 1767839950.236549 write 0x489ce6 thru 0x48bbf2 (0x1f0d bytes) 000444[0] 1767839950.257839 write 0x16e9e1 thru 0x16f181 (0x7a1 bytes) 000446[1] 1767839950.277515 mapwrite 0x70e55b thru 0x70f989 (0x142f bytes) 000447[0] 1767839950.347068 mapwrite 0x403488 thru 0x4053d0 (0x1f49 bytes) 000448[1] 1767839950.416564 read 0x52b753 thru 0x52c5e1 (0xe8f bytes) 000449[1] 1767839950.445303 write 0x9195d7 thru 0x91b300 (0x1d2a bytes) 000450[1] 1767839950.456440 mapread 0x5acb43 thru 0x5ade7f (0x133d bytes) 000451[0] 1767839950.475386 read 0x828a67 thru 0x82a704 (0x1c9e bytes) 000453[1] 1767839950.518723 mapwrite 0x20ee4a thru 0x20f20b (0x3c2 bytes) 000454[1] 1767839950.580074 write 0x64ff39 thru 0x6501e1 (0x2a9 bytes) 000455[1] 1767839950.590666 write 0x4cd814 thru 0x4ce3cf (0xbbc bytes) 000457[0] 1767839950.619283 mapwrite 0x8ef623 thru 0x8ef908 (0x2e6 bytes) 000458[1] 1767839950.707008 mapwrite 0x5c30b0 thru 0x5c47e2 (0x1733 bytes) 000459[0] 1767839950.770336 mapread 0x449cec thru 0x44b31a (0x162f bytes) 000460[0] 1767839950.807374 write 0x0f803d thru 0x0f9d4a (0x1d0e bytes) 000462[1] 1767839950.837302 write 0x74c85c thru 0x74dc63 (0x1408 bytes) 000464[1] 1767839950.871497 mapwrite 0x9597b9 thru 0x95a182 (0x9ca bytes) 000466[0] 1767839950.933232 mapwrite 0x665728 thru 0x66695d (0x1236 bytes) 000467[1] 1767839950.983610 mapread 0x7cb3cf thru 0x7cc1d6 (0xe08 bytes) 000469[1] 1767839951.016705 mapwrite 0x31f70a thru 0x32104f (0x1946 bytes) 000470[1] 1767839951.076115 trunc from 0x95a183 to 0x29cc68 (0x6bd51a bytes) 000471[0] 1767839951.110540 mapread 0x15912e thru 0x1594f2 (0x3c5 bytes) 000473[1] 1767839951.123619 write 0x16e145 thru 0x170125 (0x1fe1 bytes) 000475[0] 1767839951.138993 trunc from 0x29cc69 to 0x5fddcc (0x361164 bytes) 000477[0] 1767839951.154941 mapread 0x57bfbd thru 0x57cab2 (0xaf6 bytes) 000478[1] 1767839951.158409 write 0x44b0ea thru 0x44c2c7 (0x11de bytes) 000480[0] 1767839951.178840 mapread 0x19808a thru 0x1999a5 (0x191c bytes) 000481[1] 1767839951.189875 read 0x57276d thru 0x5732b9 (0xb4d bytes) 000482[0] 1767839951.207677 mapwrite 0x711468 thru 0x71209c (0xc35 bytes) 000483[0] 1767839951.250821 mapwrite 0x317792 thru 0x317d11 (0x580 bytes) 000484[1] 1767839951.279518 read 0x2ec3da thru 0x2ed519 (0x1140 bytes) 000485[0] 1767839951.295472 mapread 0x069c6a thru 0x06a5d9 (0x970 bytes) 000486[1] 1767839951.308640 trunc from 0x71209d to 0x8b3abd (0x1a1a21 bytes) 000488[0] 1767839951.337044 trunc from 0x8b3abe to 0x6d64c1 (0x1dd5fc bytes) 000491[0] 1767839951.388377 trunc from 0x6d64c2 to 0x903234 (0x22cd73 bytes) 000493[1] 1767839951.430950 write 0x478852 thru 0x478c6d (0x41c bytes) 000494[1] 1767839951.460335 trunc from 0x903235 to 0x45add6 (0x4a845e bytes) 000496[1] 1767839951.504538 mapread 0x30b7f1 thru 0x30c9b8 (0x11c8 bytes) 000497[1] 1767839951.523212 read 0x1c0bf7 thru 0x1c1584 (0x98e bytes) 000500[0] 1767839951.559452 read 0x20e6f8 thru 0x20f971 (0x127a bytes) All operations completed A-OK! PASS 16b (16s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 21:39:13 (1767839953) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 21:39:16 (1767839956) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.174386 s, 12.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.116219 s, 18.0 MB/s ldlm.namespaces.MGC192.168.203.158@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9e9dc54a2000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9e9dcb7c1000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9e9dc54a2000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9e9dcb7c1000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9e9dcb7c1000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9e9dcb7c1000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.50795 s, 19.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.62906 s, 64.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.01275 s, 104 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.75029 s, 38.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.55458 s, 29.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.38915 s, 75.5 MB/s PASS 16d (21s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 21:39:37 (1767839977) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.120594 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562673 s, 18.6 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 21:39:40 (1767839980) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 21:40:04 (1767840004) TIMEOUT PASS 16g (23s) == sanityn test 16i: read after truncate file ============ 21:40:27 (1767840027) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d16i.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (3s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 21:40:30 (1767840030) fail_loc=0x8000030a PASS 17 (5s) == sanityn test 18: mmap sanity check =========================================================================================== 21:40:35 (1767840035) mmap test1: basic mmap operation (PASS, 0.038125s) mmap test2: MAP_PRIVATE not write back (PASS, 0.050358s) mmap test3: concurrent mmap ops on two nodes (PASS, 5.8899s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.12s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.057643s) mmap test6: check mmap write/read content on two nodes (PASS, 0.10204s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.025s) PASS 18 (13s) == sanityn test 19: test concurrent uncached read races ========================================================================= 21:40:48 (1767840048) oleg358-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg358-client: oleg358-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 21:40:50 (1767840050) striped dir -i0 -c2 -H all_char /mnt/lustre/d20.sanityn PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 21:40:53 (1767840053) striped dir -i1 -c2 -H all_char /mnt/lustre/d21.sanityn rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (3s) == sanityn test 23: others should see updated atime while another read============================================================== 21:40:56 (1767840056) now is 1767840057 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 new atime is 1767840118 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 21:42:00 (1767840120) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 483.8K 383 483.4K 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 484.9K 336 484.6K 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 115.3K 438 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.4K 442 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 483.8K 383 483.4K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 484.9K 336 484.6K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 115.3K 438 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.4K 442 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.6M 2.1G 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495416 383 495033 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 496533 336 496197 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 118102 438 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118138 442 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 236079 719 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 483.8K 383 483.4K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 484.9K 336 484.6K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 115.3K 438 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.4K 442 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495416 383 495033 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 496533 336 496197 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 118102 438 117664 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118138 442 117696 1% /mnt/lustre[OST:1] filesystem_summary: 236079 719 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495416 383 495033 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 496533 336 496197 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 118102 438 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118138 442 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 236079 719 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 21:42:03 (1767840123) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 21:42:05 (1767840125) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 21:42:08 (1767840128) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] PASS 25b (3s) == sanityn test 26a: allow mtime to get older ============ 21:42:11 (1767840131) utime: good mknod times 1767840131 <= 1767840132 <= 1767840132 for /mnt/lustre/f26a utime: good mknod times 1767840131 <= 1767840132 <= 1767840132 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (4s) == sanityn test 26b: sync mtime between ost and mds ====== 21:42:15 (1767840135) PASS 26b (5s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 21:42:22 (1767840142) striped dir -i0 -c2 -H crush /mnt/lustre/d30.sanityn 0 PASS 30 (6s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 21:42:27 (1767840147) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d31a.sanityn fail_loc=0x314 PASS 31a (4s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 21:42:31 (1767840151) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg358-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d31b.sanityn fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.158@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (21s) == sanityn test 31r: open-rename(replace) race =========== 21:42:52 (1767840172) -rw-r--r-- 1 root root 0 Jan 7 21:42 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 21:42:58 (1767840178) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 21:43:00 (1767840180) Stopping /mnt/lustre-mds1 (opts:) on oleg358-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 sync_count 2 sync_count 0 PASS 33c (22s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 21:43:22 (1767840202) CoS count 2 CoS count 2 CoS count 2 CoS count 2 CoS count 2 CoS count 4 CoS count 3 CoS count 4 PASS 33d (47s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 21:44:09 (1767840249) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 21:44:10 (1767840250) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e9dc54a2000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9e9dcb7c1000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff9e9dc54a2000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff9e9dcb7c1000.ost_server_uuid in FULL state after 0 sec oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9e9dc54a2000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9e9dcb7c1000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff9e9dc54a2000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff9e9dcb7c1000.ost_server_uuid in FULL state after 0 sec oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e9dc54a2000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9e9dcb7c1000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff9e9dc54a2000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9e9dcb7c1000.ost_server_uuid in IDLE state after 0 sec oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9e9dc54a2000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9e9dcb7c1000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff9e9dc54a2000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff9e9dcb7c1000.ost_server_uuid in FULL state after 0 sec oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e9dc54a2000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9e9dcb7c1000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff9e9dc54a2000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9e9dcb7c1000.ost_server_uuid in IDLE state after 0 sec oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9e9dc54a2000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9e9dcb7c1000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff9e9dc54a2000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff9e9dcb7c1000.ost_server_uuid in FULL state after 0 sec PASS 34 (87s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 21:45:37 (1767840337) striped dir -i1 -c2 -H all_char /mnt/lustre/d35.sanityn fail_loc=0x80000317 Race attempt 0 Wait for 52311 52440 for 60 sec... ls: closing directory '/mnt/lustre/d35.sanityn': Interrupted system call - open/close 2318 (time 1767840350.71 total 10.00 last 231.73) total: 4000 open/close in 17.45 seconds: 229.19 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.158@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre 192.168.203.158@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 21:46:43 (1767840403) striped dir -i0 -c2 -H crush /mnt/lustre/d36.sanityn Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg358-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.378326 s, 139 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.368309 s, 142 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.380511 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.350785 s, 149 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.313596 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.37044 s, 142 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.380114 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.388149 s, 135 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.347418 s, 151 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.376244 s, 139 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.361946 s, 145 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7525376) after_dd(7474176) after(7525376) PASS 36 (236s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 21:50:39 (1767840639) striped dir -i1 -c2 -H all_char /mnt/lustre/d37.sanityn multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 - create 8053 (time 1767840650.89 total 10.01 last 804.84) total: 10000 create in 12.65 seconds: 790.27 ops/second PASS 37 (16s) == sanityn test 39a: test from 11063 ============================================================================================ 21:50:56 (1767840656) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 21:50:58 (1767840658) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 21:51:01 (1767840661) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 21:51:04 (1767840664) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 21:51:06 (1767840666) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 21:51:14 (1767840674) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 21:51:22 (1767840682) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=15 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 21:51:30 (1767840690) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 21:51:38 (1767840698) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=11 No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 21:51:45 (1767840705) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (6s) == sanityn test 41b: pdirops: create vs create ======================================================================== 21:51:51 (1767840711) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 21:51:56 (1767840716) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 21:52:01 (1767840721) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 21:52:06 (1767840726) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 21:52:11 (1767840731) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 21:52:16 (1767840736) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 21:52:21 (1767840741) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 21:52:26 (1767840746) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (723s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 22:04:29 (1767841469) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 22:04:35 (1767841475) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 22:04:40 (1767841480) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 22:04:45 (1767841485) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 22:04:50 (1767841490) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 22:04:55 (1767841495) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 22:05:00 (1767841500) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 22:05:05 (1767841505) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42h (9s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 22:05:14 (1767841514) PASS 43a (96s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 22:06:50 (1767841610) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 43b (8s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 22:06:58 (1767841618) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43c (7s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 22:07:05 (1767841625) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (8s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 22:07:13 (1767841633) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43e (7s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 22:07:20 (1767841640) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (7s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 22:07:27 (1767841647) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (7s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 22:07:34 (1767841654) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43h (7s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 22:07:42 (1767841662) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43i (6s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 22:07:48 (1767841668) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (63s) == sanityn test 43k: unlink vs create ==================== 22:08:51 (1767841731) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (658s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 22:19:49 (1767842389) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=2 mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Jan 7 22:19:53 EST 2026 PASS 44a (6s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 22:19:55 (1767842395) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 44b (6s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 22:20:01 (1767842401) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (6s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 22:20:07 (1767842407) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44d (6s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 22:20:13 (1767842413) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44e (6s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 22:20:19 (1767842419) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=11 Conflict PASS 44f (6s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 22:20:25 (1767842425) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=10 Conflict PASS 44g (6s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 22:20:31 (1767842431) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44h (7s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 22:20:38 (1767842438) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f44i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f44i.sanityn': File exists Conflict PASS 44i (6s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 22:20:45 (1767842445) PASS 45a (61s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 22:21:46 (1767842506) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 45b (6s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 22:21:52 (1767842512) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 22:21:58 (1767842518) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=12 rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 22:22:03 (1767842523) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 Conflict PASS 45e (6s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 22:22:09 (1767842529) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=11 mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 22:22:14 (1767842534) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (6s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 22:22:20 (1767842540) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 22:22:25 (1767842545) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45i (6s) == sanityn test 45j: read vs rename ====================== 22:22:31 (1767842551) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (482s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 22:30:33 (1767843033) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=3 mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 22:30:38 (1767843038) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 22:30:43 (1767843043) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 22:30:48 (1767843048) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=7 Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 22:30:53 (1767843053) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 22:30:58 (1767843058) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=11 Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 22:31:04 (1767843064) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46g (6s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 22:31:09 (1767843069) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 22:31:14 (1767843074) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f46i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f46i.sanityn': File exists Conflict PASS 46i (5s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 22:31:19 (1767843079) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f47a.sanityn': File exists Conflict PASS 47a (5s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 22:31:24 (1767843084) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 open: File exists Conflict PASS 47b (6s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 22:31:30 (1767843090) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f47c.sanityn' to '/mnt/lustre2/f47c.sanityn-2': File exists Conflict PASS 47c (6s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 22:31:36 (1767843096) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 47d (5s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 22:31:41 (1767843101) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f47e.sanityn' with non-directory Conflict PASS 47e (6s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 22:31:47 (1767843107) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 47f (5s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 22:31:52 (1767843112) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47g (6s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 22:31:58 (1767843118) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00457266 s, 2.2 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 22:32:05 (1767843125) 0+1 records in 0+1 records out 158 bytes copied, 0.00321883 s, 49.1 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (263734) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 22:32:09 (1767843129) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00275623 s, 372 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153117 s, 68.5 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 22:32:23 (1767843143) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00360591 s, 284 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 22:32:29 (1767843149) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980031 s, 107 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 22:32:32 (1767843152) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1259 0x4eb 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1259 0x4eb 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 22:32:36 (1767843156) llite.lustre-ffff9e9dc54a2000.xattr_cache=1 llite.lustre-ffff9e9dcb7c1000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281041 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:32:37.000000000 -0500 Modify: 2026-01-07 22:32:37.000000000 -0500 Change: 2026-01-07 22:32:37.000000000 -0500 Birth: 2026-01-07 22:32:37.000000000 -0500 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762674 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:32:42.000000000 -0500 Modify: 2026-01-07 22:32:43.000000000 -0500 Change: 2026-01-07 22:32:43.000000000 -0500 Birth: 2026-01-07 22:32:43.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281044 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:32:48.000000000 -0500 Modify: 2026-01-07 22:32:48.000000000 -0500 Change: 2026-01-07 22:32:48.000000000 -0500 Birth: 2026-01-07 22:32:48.000000000 -0500 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762675 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:32:48.000000000 -0500 Modify: 2026-01-07 22:32:48.000000000 -0500 Change: 2026-01-07 22:32:48.000000000 -0500 Birth: 2026-01-07 22:32:48.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281045 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:32:53.000000000 -0500 Modify: 2026-01-07 22:32:54.000000000 -0500 Change: 2026-01-07 22:32:54.000000000 -0500 Birth: 2026-01-07 22:32:54.000000000 -0500 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762677 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:32:53.000000000 -0500 Modify: 2026-01-07 22:32:54.000000000 -0500 Change: 2026-01-07 22:32:54.000000000 -0500 Birth: 2026-01-07 22:32:54.000000000 -0500 PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 22:33:00 (1767843180) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 22:33:08 (1767843188) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 22:33:16 (1767843196) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 22:33:28 (1767843208) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 60: Verify data_version behaviour ======== 22:33:38 (1767843218) striped dir -i0 -c2 -H all_char /mnt/lustre/d60.sanityn PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 22:33:40 (1767843220) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 22:33:42 (1767843222) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 22:33:44 (1767843224) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00300902 s, 21.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 22:33:46 (1767843226) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00475429 s, 13.8 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 22:33:47 (1767843227) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 22:33:48 (1767843228) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00486915 s, 13.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 22:33:49 (1767843229) llite.lustre-ffff9e9dc54a2000.xattr_cache=1 llite.lustre-ffff9e9dcb7c1000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 22:33:51 (1767843231) llite.lustre-ffff9e9dc54a2000.xattr_cache=1 llite.lustre-ffff9e9dcb7c1000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 22:33:54 (1767843234) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00206276 s, 496 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00524396 s, 195 kB/s 273908: get lock2 273910: get lock1 273908: done 273908: get lock1 273910: done 273910: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 273908: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 273910: cannot get lock2: Resource deadlock avoided 273910: exit rc=0 273908: done 273908: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 22:34:00 (1767843240) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0186075 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101886 s, 103 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 274491 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 22:34:07 (1767843247) striped dir -i0 -c2 -H crush /mnt/lustre/d76.sanityn opening files: ................ closing files: ................ PASS 76 (45s) == sanityn test 77a: check FIFO NRS policy =============== 22:34:52 (1767843292) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.105542 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056801 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685519 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806318 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802763 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841943 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739304 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647618 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687902 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999662 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854394 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836802 s, 12.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479598 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1102 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419335 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097497 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864953 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209983 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166682 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203546 s, 51.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267494 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134044 s, 78.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277385 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197956 s, 53.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121886 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270744 s, 38.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292026 s, 35.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704275 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161332 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153804 s, 68.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162883 s, 64.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283918 s, 36.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163692 s, 64.1 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 22:34:55 (1767843295) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.116567 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197518 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0452094 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473557 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420004 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517427 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482363 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760833 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0565227 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715832 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105113 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963344 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110647 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121263 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107791 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10034 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110559 s, 9.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248446 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220675 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106016 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00426097 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207387 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318174 s, 33.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152471 s, 68.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324678 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336891 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210486 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00415546 s, 252 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316827 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100492 s, 104 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142311 s, 73.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178265 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214939 s, 48.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.109276 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397907 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575344 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0570682 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.069558 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843245 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920456 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742979 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764384 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107997 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989861 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112158 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987132 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110003 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102225 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102562 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112607 s, 9.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133665 s, 78.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119687 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227551 s, 46.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103586 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110548 s, 94.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615953 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239212 s, 43.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177792 s, 59.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294332 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024063 s, 43.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283431 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226986 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00379924 s, 276 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236618 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010574 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542933 s, 193 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 22:34:59 (1767843299) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.116957 s, 143 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359735 s, 29.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426847 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360705 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705953 s, 14.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713522 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108691 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975063 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0959275 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983869 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131143 s, 8.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100488 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137956 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111276 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130239 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121866 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153752 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208947 s, 50.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197509 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296293 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248428 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312117 s, 33.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303533 s, 34.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271265 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366512 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363671 s, 28.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111747 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119293 s, 87.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386946 s, 27.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00280849 s, 373 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195941 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174178 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307098 s, 34.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.119814 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338537 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0463065 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608683 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062755 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924713 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832922 s, 12.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656473 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912754 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999411 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0996219 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992314 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09508 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854049 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099472 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109633 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105545 s, 9.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138234 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193187 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173589 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141994 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165797 s, 63.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00418254 s, 251 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208442 s, 50.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183578 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255428 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235752 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224216 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00448722 s, 234 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01169 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233114 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00321485 s, 326 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138559 s, 75.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.122731 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225108 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082315 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804809 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830301 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101544 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901233 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687213 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771604 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758641 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881202 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110928 s, 9.5 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965505 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130818 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113116 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12001 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126422 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224787 s, 46.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244336 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221301 s, 47.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219905 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236348 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020684 s, 50.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208088 s, 50.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018381 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772093 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260106 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158813 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416317 s, 252 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252865 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107288 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796148 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00262212 s, 400 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 22:35:05 (1767843305) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.130696 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632564 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753618 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861666 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819403 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094391 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964243 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124076 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755509 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142887 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145203 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126301 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136837 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145345 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126619 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125127 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123973 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145323 s, 72.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213085 s, 49.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257838 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222932 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164688 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216578 s, 48.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334129 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259047 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212268 s, 49.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383386 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294945 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323753 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299925 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305906 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293056 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315058 s, 33.3 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.127395 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488493 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667389 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108406 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120651 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136722 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116875 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152035 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143898 s, 7.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14062 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133873 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167699 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154087 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15226 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165543 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148774 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16413 s, 6.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354178 s, 29.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302355 s, 34.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321115 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316603 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038524 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371343 s, 28.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280157 s, 37.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342997 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375783 s, 27.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181125 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290628 s, 36.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370411 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034657 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350226 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037451 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786963 s, 133 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.185999 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465789 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554574 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662307 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865726 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999265 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936085 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832015 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105313 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116021 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128407 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123082 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148218 s, 7.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109919 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925238 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115843 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158161 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217855 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160933 s, 65.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750066 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223619 s, 46.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129739 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107479 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301589 s, 34.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971012 s, 108 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931348 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021037 s, 49.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198133 s, 52.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671466 s, 156 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132193 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141524 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158016 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112524 s, 93.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) == sanityn test 77e: check TBF NID nrs policy ============ 22:35:10 (1767843310) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.58@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.114388 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024061 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395157 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246305 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376585 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934943 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.081935 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861126 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694599 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562912 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0716984 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103383 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0914636 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894384 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157429 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117251 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165803 s, 6.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401914 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456784 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516294 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544601 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0464947 s, 22.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453255 s, 23.1 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448349 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540205 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558678 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456372 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510903 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521102 s, 20.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500078 s, 21.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582194 s, 18.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050276 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700783 s, 15.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.121451 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666706 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797012 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809105 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651557 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862792 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784885 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116927 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1131 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136721 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144378 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122791 s, 8.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12614 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14819 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129867 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144041 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15939 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00449733 s, 233 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172944 s, 60.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152743 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160094 s, 65.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172734 s, 60.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183468 s, 57.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156583 s, 67.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199398 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248475 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00466087 s, 225 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255452 s, 41.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888036 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564859 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416675 s, 252 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271951 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00540582 s, 194 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.118929 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250362 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281263 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449829 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056073 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558704 s, 18.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635073 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648443 s, 16.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616688 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888276 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537852 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578292 s, 18.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108248 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098426 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925761 s, 11.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100829 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126358 s, 8.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183096 s, 57.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153802 s, 68.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188787 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116693 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184372 s, 56.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016575 s, 63.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253509 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188252 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012579 s, 83.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243981 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00459637 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248346 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130609 s, 80.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156239 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708355 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402716 s, 260 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.143248 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323425 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470953 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581601 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0486918 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404521 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846683 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894092 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847977 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076732 s, 13.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425672 s, 24.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787359 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834356 s, 12.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879045 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820764 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791009 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879055 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215248 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256892 s, 40.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240292 s, 43.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214799 s, 48.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544392 s, 193 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329345 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317515 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148555 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380554 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191369 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111647 s, 93.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586699 s, 179 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032991 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175514 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196546 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014848 s, 70.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 22:35:19 (1767843319) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.131503 s, 128 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340657 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042453 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526112 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525174 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334231 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587962 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104859 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893079 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789751 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975317 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939353 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885895 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0974215 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734223 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821707 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849152 s, 12.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140257 s, 74.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171829 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185926 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163306 s, 64.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722673 s, 145 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136588 s, 76.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217978 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169824 s, 61.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653275 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00470338 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025236 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0047299 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012339 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005242 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109413 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166711 s, 62.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.128938 s, 130 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309212 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458877 s, 22.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918678 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995582 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957011 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086726 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820389 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933608 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0951453 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840284 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813956 s, 12.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105845 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869101 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658038 s, 15.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0959918 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103288 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103004 s, 102 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191119 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219878 s, 47.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192289 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193354 s, 54.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189369 s, 55.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235493 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226365 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268204 s, 39.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559297 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230648 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024333 s, 43.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303228 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936722 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116523 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00485592 s, 216 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.173497 s, 96.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442448 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053717 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520257 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831747 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619631 s, 16.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510337 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586432 s, 17.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957167 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853077 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124342 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140657 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138233 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125574 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157462 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153002 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151168 s, 6.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157352 s, 66.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260867 s, 40.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301744 s, 34.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217232 s, 48.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029851 s, 35.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333003 s, 31.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165666 s, 63.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273329 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326728 s, 32.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263452 s, 39.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123587 s, 84.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00468616 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02145 s, 48.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336048 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014084 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185335 s, 56.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.133398 s, 126 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454871 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618955 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735729 s, 14.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949498 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982883 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132773 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132152 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133213 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120482 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14513 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111187 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136762 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128333 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154381 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130128 s, 8.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148675 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196767 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181086 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172894 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258766 s, 40.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288561 s, 36.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154375 s, 67.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287481 s, 36.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100601 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034792 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232844 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262149 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627989 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909277 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717519 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626542 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750672 s, 140 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (8s) == sanityn test 77g: Change TBF type directly ============ 22:35:27 (1767843327) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 22:35:31 (1767843331) oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg358-client: oleg358-server: ssh exited with exit code 2 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 PASS 77h (5s) == sanityn test 77i: Change rank of TBF rule ============= 22:35:35 (1767843335) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (8s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 22:35:43 (1767843343) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.113804 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388655 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480012 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643289 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075372 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682887 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660255 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796603 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943369 s, 11.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726815 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669107 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864278 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927815 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802696 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593446 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843056 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09275 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165782 s, 63.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021637 s, 48.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250569 s, 41.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273062 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259067 s, 40.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175978 s, 59.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300722 s, 34.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219959 s, 47.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279277 s, 37.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194416 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356065 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343581 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330133 s, 31.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221962 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216863 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010198 s, 103 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94091 s, 21.2 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4339 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 22:36:32 (1767843392) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.654148 s, 25.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147778 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173342 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166391 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166486 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192879 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187502 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182186 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180767 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352052 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371613 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359512 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382015 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.374508 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399152 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393647 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.38852 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132557 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134388 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1392 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149049 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143081 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146297 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149324 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136384 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147735 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135735 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138092 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142737 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138101 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151389 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145904 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145195 s, 7.2 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6679 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9518 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.651461 s, 25.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15255 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159184 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167711 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192814 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183089 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177706 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205294 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200298 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353896 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352149 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359027 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35075 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3422 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349682 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361873 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353921 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150685 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151599 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152026 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153532 s, 6.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153132 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146857 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150302 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15754 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155463 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153962 s, 6.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153272 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159664 s, 6.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155932 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15732 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164388 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161047 s, 6.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6715 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9579 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (139s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 22:38:51 (1767843531) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.198412 s, 84.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199215 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268211 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028683 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560914 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212705 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450638 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310413 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356535 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601396 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954516 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699286 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897624 s, 11.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828158 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838207 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908047 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913328 s, 11.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014877 s, 70.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122076 s, 85.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018949 s, 55.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199837 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117076 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252931 s, 41.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239003 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224235 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214276 s, 48.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01044 s, 100 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170203 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698477 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243506 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246707 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00330842 s, 317 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00445249 s, 236 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93134 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73274 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.58@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.58@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.155574 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483855 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375758 s, 27.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526628 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855352 s, 12.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775053 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948711 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919969 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880515 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789667 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875263 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135329 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147162 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222098 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218999 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335283 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.345058 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138334 s, 75.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156622 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178847 s, 58.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014022 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160619 s, 65.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166574 s, 62.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178316 s, 58.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166495 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115341 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246819 s, 42.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00366414 s, 286 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00346039 s, 303 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955109 s, 110 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244025 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00290265 s, 361 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941241 s, 111 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94564 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73468 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.58@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.157468 s, 107 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0351509 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0519283 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607591 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610628 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685048 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800323 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618503 s, 17.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802908 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087754 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801296 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098055 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842414 s, 12.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909547 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967592 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813302 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978033 s, 10.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171135 s, 61.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187828 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171236 s, 61.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143077 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172976 s, 60.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155707 s, 67.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173241 s, 60.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018126 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141026 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921989 s, 114 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235112 s, 44.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696609 s, 151 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888531 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252919 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00474797 s, 221 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00189631 s, 553 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93561 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.88082 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.255087 s, 65.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052788 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562126 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918354 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861571 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823552 s, 12.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843098 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782756 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770464 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141758 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147422 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277561 s, 3.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277273 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326127 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330997 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347956 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326851 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176498 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175927 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209396 s, 50.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163677 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159778 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195119 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210939 s, 49.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00434342 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134233 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224386 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637183 s, 165 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243391 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214921 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256705 s, 40.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246578 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981698 s, 107 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78361 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73365 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93412 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73556 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.65707 s, 25.5 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523834 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709564 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0595486 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702728 s, 14.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792645 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784235 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725174 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606537 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338733 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334737 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356809 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.360729 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371642 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354164 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387606 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37927 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197255 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174218 s, 60.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219417 s, 47.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116775 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243643 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195676 s, 53.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260636 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231054 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272849 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102857 s, 102 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294071 s, 35.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032856 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264198 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00312664 s, 335 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288858 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00250911 s, 418 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6765 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4313 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.204515 s, 82.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342563 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442096 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733451 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658408 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681457 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077229 s, 13.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070222 s, 14.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625595 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946791 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10052 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937032 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887412 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845238 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109093 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981058 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928344 s, 11.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195781 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151716 s, 69.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853917 s, 123 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162202 s, 64.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193584 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203538 s, 51.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193187 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105076 s, 99.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025539 s, 41.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290151 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00382112 s, 274 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253519 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250574 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611268 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057411 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00217707 s, 482 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92413 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73566 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.352961 s, 47.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246624 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026482 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420445 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047526 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0501359 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575357 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487548 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0452118 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152817 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152487 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171078 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167268 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332675 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322311 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.389412 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409221 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00515207 s, 204 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181357 s, 57.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123064 s, 85.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243236 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286639 s, 36.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022121 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00478396 s, 219 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234626 s, 44.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273753 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248446 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276862 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245964 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114968 s, 91.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239906 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315163 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00252306 s, 416 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77877 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73103 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75651 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72756 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (359s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 22:44:50 (1767843890) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 22:44:54 (1767843894) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg358-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.61536 s, 290 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg358-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0163 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 22:45:44 (1767843944) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.112794 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296527 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049541 s, 21.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612363 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594598 s, 17.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534311 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740198 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071893 s, 14.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578265 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0512497 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983548 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964828 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864874 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759672 s, 13.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768418 s, 13.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670496 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10137 s, 10.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9e9dc54a2000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9e9dc54a2000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170536 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139293 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170825 s, 61.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119381 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186806 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00397286 s, 264 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230248 s, 45.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235996 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00357398 s, 293 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124053 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00436217 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232794 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198694 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024492 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025238 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806388 s, 130 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg358-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg358-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93628 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96809 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.199904 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328839 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582461 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063629 s, 16.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084625 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804895 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809815 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911679 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091048 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988193 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108431 s, 9.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136759 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118624 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234914 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223508 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339278 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325579 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120826 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013584 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136917 s, 76.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221044 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176502 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180889 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219687 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196354 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177759 s, 59.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266323 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218007 s, 48.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353169 s, 29.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104151 s, 101 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242074 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00422496 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117433 s, 89.3 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg358-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg358-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.98224 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96979 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (73s) == sanityn test 77o: Changing rank should not panic ====== 22:46:57 (1767844017) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 22:47:01 (1767844021) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (37s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 22:47:38 (1767844058) mds.MDS.mdt.nrs_policies=tbf oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg358-client: oleg358-server: ssh exited with exit code 36 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (12s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 22:47:50 (1767844070) oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg358-client: oleg358-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 22:47:53 (1767844073) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d79.sanityn File: /mnt/lustre/d79.sanityn Size: 12288 Blocks: 24 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762994 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:47:54.000000000 -0500 Modify: 2026-01-07 22:47:54.000000000 -0500 Change: 2026-01-07 22:47:54.000000000 -0500 Birth: 2026-01-07 22:47:54.000000000 -0500 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 22:48:06 (1767844086) total: 10 open/close in 0.03 seconds: 301.76 ops/second multiop /mnt/lustre2/d80a.sanityn/f80a.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 lfs migrate: /mnt/lustre/d80a.sanityn/f80a.sanityn migrate failed: Device or resource busy (16) Finish migration, then checking.. PASS 80a (4s) == sanityn test 80b: Accessing directory during migration ========================================================== 22:48:10 (1767844090) start migration thread 303354 accessing the migrating directory for 1 minute... concurrent access 147 failures, 2406 successes /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 303354 Killed ( while true; do mdt_idx=$((RANDOM % MDSCOUNT)); $LFS migrate -m $mdt_idx $migdir1 &> /dev/null; done ) (wd: ~) PASS 80b (62s) == sanityn test 81a: rename and stat under striped directory ========================================================== 22:49:12 (1767844152) stat: cannot statx 'd0/0': No such file or directory File: d0/0 Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289282298 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 22:49:13.000000000 -0500 Modify: 2026-01-07 22:49:13.000000000 -0500 Change: 2026-01-07 22:49:13.000000000 -0500 Birth: 2026-01-07 22:49:13.000000000 -0500 PASS 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 22:49:14 (1767844154) total: 1000 open/close in 2.19 seconds: 456.24 ops/second /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 307532 Killed ( while true; do touch $DIR1/$tdir; done ) (wd: ~) PASS 81b (28s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 22:49:42 (1767844182) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 22:49:43 (1767844183) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7847 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7847 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 22:49:45 (1767844185) start pid 310771 to create/unlink striped directory start pid 310772 to stat striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4949: 310771 Terminated ( cd $DIR1; while true; do $LFS mkdir -i1 -c2 $tdir; rmdir $tdir; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4949: 310772 Terminated ( cd $DIR2; while true; do stat $tdir > /dev/null 2>&1; done ) PASS 83 (122s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 22:51:47 (1767844307) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 22:51:54 (1767844314) start pid 314512 to open/create under striped directory start pid 314513 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5011: 314512 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; touch $tdir/f{0..3} > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5011: 314513 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 90 (182s) == sanityn test 91: chmod and unlink striped directory === 22:54:56 (1767844496) start pid 321644 to chmod striped directory start pid 321645 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5048: 321644 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; chmod go+w $tdir > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5048: 321645 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 91 (182s) == sanityn test 92: create remote directory under orphan directory ========================================================== 22:57:58 (1767844678) fail_loc=0x1408 mkdir: cannot create directory '/mnt/lustre2/d92.sanityn/dir': No such file or directory lfs setdirstripe: cannot resolve path '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory (2) lfs setdirstripe: '/mnt/lustre2/d92.sanityn/remote_dir' is not on a Lustre filesystem: No such file or directory (2) lfs setdirstripe: cannot create dir '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory fail_loc=0 PASS 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 22:58:00 (1767844680) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2575 0xa0f 0 0 2582 0xa16 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2576 0xa10 0 0 2583 0xa17 0 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 22:58:09 (1767844689) osc.lustre-OST0000-osc-ffff9e9dc54a2000.idle_timeout=0 osc.lustre-OST0000-osc-ffff9e9dcb7c1000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9e9dc54a2000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9e9dcb7c1000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0429492 s, 191 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09026 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 331000 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0109879 s, 373 kB/s osc.lustre-OST0000-osc-ffff9e9dc54a2000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9e9dcb7c1000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9e9dc54a2000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9e9dcb7c1000.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 22:58:19 (1767844699) llite.lustre-ffff9e9dc54a2000.fast_read=0 llite.lustre-ffff9e9dcb7c1000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.0012391 s, 9.9 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff9e9dc54a2000.fast_read=1 llite.lustre-ffff9e9dcb7c1000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 22:58:27 (1767844707) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 22:58:28 (1767844708) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00126382 s, 3.2 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 22:58:30 (1767844710) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0275599 s, 76.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 22:58:32 (1767844712) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0192769 s, 109 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 22:58:34 (1767844714) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.0018007 s, 6.7 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 22:58:36 (1767844716) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000472735 s, 8.7 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 22:58:38 (1767844718) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000419763 s, 9.8 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 22:58:40 (1767844720) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000447163 s, 9.2 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 22:58:44 (1767844724) striped dir -i1 -c2 -H crush /mnt/lustre/d103.sanityn fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1767844725 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1767844728 Finishing test test23 at 1767844728 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 22:58:51 (1767844731) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 22:58:52 (1767844732) striped dir -i1 -c2 -H all_char /mnt/lustre/d105.sanityn fail_loc=0x416 PASS 105 (79s) == sanityn test 106a: Verify the btime via statx() ======= 23:00:11 (1767844811) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 23:00:12 (1767844812) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00580136 s, 177 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289285347 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 23:00:13.000000000 -0500 Modify: 2026-01-07 23:00:13.000000000 -0500 Change: 2026-01-07 23:00:13.000000000 -0500 Birth: 2026-01-07 23:00:13.000000000 -0500 /mnt/lustre/f106b.sanityn 144115205289285347 -rw-r--r-- 1 0 0 1767844813 1767844813 1767844813 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289285347 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 23:00:13.000000000 -0500 Modify: 2026-01-07 23:00:13.000000000 -0500 Change: 2026-01-07 23:00:13.000000000 -0500 Birth: 2026-01-07 23:00:13.000000000 -0500 1767844813 1024 1 PASS 106b (3s) == sanityn test 106c: Verify statx attributes mask ======= 23:00:15 (1767844815) PASS 106c (2s) == sanityn test 107a: Basic grouplock conflict =========== 23:00:17 (1767844817) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.097288 s, 108 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2581 0xa15 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 23:00:21 (1767844821) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0799333 s, 131 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2587 0xa1b 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7847 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 23:00:30 (1767844830) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00247366 s, 4.0 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.31363 s, 1.9 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 23:00:36 (1767844836) 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg358-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre2 PASS 109 (579s) == sanityn test 110: do not grant another lock on resend ========================================================== 23:10:15 (1767845415) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934275 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-07 23:10:15.000000000 -0500 Modify: 2026-01-07 23:10:15.000000000 -0500 Change: 2026-01-07 23:10:15.000000000 -0500 Birth: 2026-01-07 23:10:16.000000000 -0500 fail_loc=0 fail_val=0 Connected clients: oleg358-client.virtnet oleg358-client.virtnet PASS 110 (44s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 23:10:59 (1767845459) fail_loc=0x8000018a /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5892: 400067 Terminated $MULTIOP $DIR2/$tdir/mdt0dir/foodir/file2 Ow4096_c PASS 111 (6s) == sanityn test 112: update max-inherit in default LMV === 23:11:05 (1767845465) lmv.lustre-clilmv-ffff9e9dc66b9800.qos_threshold_rr=90 lmv.lustre-clilmv-ffff9e9dd0312000.qos_threshold_rr=90 PASS 112 (5s) == sanityn test 113: check servers of specified fs ======= 23:11:10 (1767845470) lustre-OST0000-osc-ffff9e9dc66b9800 active. lustre-OST0001-osc-ffff9e9dc66b9800 active. lustre-OST0000-osc-ffff9e9dc66b9800 active. lustre-OST0001-osc-ffff9e9dc66b9800 active. MGC192.168.203.158@tcp active. PASS 113 (3s) cleanup: ====================================================== == sanityn test complete, duration 5852 sec ============== 23:11:13 (1767845473) Stopping clients: oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 opts: