-----============= acceptance-small: sanityn ============----- Thu Apr 4 20:05:23 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 20:05:25 (1712275525) === Starting client oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Started clients oleg432-client.virtnet: 192.168.204.132@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg432-client.virtnet: executing check_config_client /mnt/lustre oleg432-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg432-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800a9c2d800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012a4ae000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9c2d800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a4ae000.idle_timeout=debug disable quota as required oleg432-server: oleg432-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 20:05:32 (1712275532) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138235 s, 75.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6742] debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 20:05:33 (1712275533) PASS 0 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 20:05:36 (1712275536) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 20:05:39 (1712275539) -rw-r--r-- 1 root root 0 Apr 4 20:05 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 20:05:42 (1712275542) -rw-r--r-- 1 root root 0 Apr 4 20:05 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 20:05:45 (1712275545) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 20:05:48 (1712275548) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2e: check chmod on root is propagated to others ========================================================== 20:05:51 (1712275551) total 1 -rwxrwxrwx 1 root root 0 Apr 4 20:05 f2a -rwxrwxrwx 1 root root 0 Apr 4 20:05 f2b total 1 -rwxrwxrwx 1 root root 0 Apr 4 20:05 f2a -rwxrwxrwx 1 root root 0 Apr 4 20:05 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00251362 s, 204 kB/s PASS 2e (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 20:05:54 (1712275554) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2g: check blocks update on sync write ==== 20:05:57 (1712275557) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.100717 s, 20.8 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 20:06:00 (1712275560) PASS 3 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 4: fstat validation on multiple mount points ==================================================================== 20:06:03 (1712275563) PASS 4 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 20:06:06 (1712275566) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 6: remove of open file on other node ============================================================================ 20:06:09 (1712275569) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 7: remove of open directory on other node ======================================================================= 20:06:12 (1712275572) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 8: remove of open special file on other node ==================================================================== 20:06:15 (1712275575) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 20:06:18 (1712275578) PASS 9a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9b: append to striped sparse file ======== 20:06:20 (1712275580) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932741 s, 112 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.00763048 s, 0.4 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 20:06:23 (1712275583) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0012107 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00314702 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00323578 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00303214 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00239382 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00199156 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00306274 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00326229 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00241661 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00270045 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00231935 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00266881 s, 0.4 kB/s PASS 10a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 20:06:27 (1712275587) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00428548 s, 717 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00622076 s, 658 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000197741 s, 15.5 MB/s PASS 10b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 11: execution of file opened for write should return error ============================================================== 20:06:30 (1712275590) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 20:06:33 (1712275593) warning: '-runtime' deprecated, use '-t runtime' instead - link 3023 (time 1712275603.85 total 10.00 last 302.17) - link 5571 (time 1712275613.85 total 20.01 last 254.72) - link 8569 (time 1712275623.85 total 30.01 last 299.77) - link 10000 (time 1712275628.30 total 34.46 last 321.53) - link 13073 (time 1712275638.31 total 44.46 last 307.23) - link 16076 (time 1712275648.31 total 54.46 last 300.28) - link 19237 (time 1712275658.31 total 64.46 last 316.09) - link 20000 (time 1712275661.12 total 67.28 last 270.99) - link 22741 (time 1712275671.13 total 77.28 last 274.02) - link 25878 (time 1712275681.13 total 87.28 last 313.64) - link 29185 (time 1712275691.13 total 97.28 last 330.68) total: 29967 link in 100.00 seconds: 299.67 ops/second using seed 3632309148 running for 100 seconds - stat 10000 (time 1712275598 ; total 4 ; last 4) - stat 20000 (time 1712275601 ; total 7 ; last 3) - stat 30000 (time 1712275604 ; total 10 ; last 3) - stat 40000 (time 1712275608 ; total 14 ; last 4) - stat 50000 (time 1712275611 ; total 17 ; last 3) - stat 60000 (time 1712275615 ; total 21 ; last 4) - stat 70000 (time 1712275618 ; total 24 ; last 3) - stat 80000 (time 1712275621 ; total 27 ; last 3) - stat 90000 (time 1712275624 ; total 30 ; last 3) - stat 100000 (time 1712275628 ; total 34 ; last 4) - stat 110000 (time 1712275631 ; total 37 ; last 3) - stat 120000 (time 1712275634 ; total 40 ; last 3) - stat 130000 (time 1712275637 ; total 43 ; last 3) - stat 140000 (time 1712275640 ; total 46 ; last 3) - stat 150000 (time 1712275644 ; total 50 ; last 4) - stat 160000 (time 1712275652 ; total 58 ; last 8) - stat 170000 (time 1712275662 ; total 68 ; last 10) - stat 179814 (time 1712275673 ; total 79 ; last 11) - stat 180000 (time 1712275673 ; total 79 ; last 0) - stat 190000 (time 1712275682 ; total 88 ; last 9) - stat 200000 (time 1712275691 ; total 97 ; last 9) total: 202021 stats in 100 seconds: 2020.209961 stats/second - unlinked 0 (time 1712275644 ; total 0 ; last 0) - unlinked 10000 (time 1712275695 ; total 51 ; last 51) - unlinked 20000 (time 1712275717 ; total 73 ; last 22) unlink(/mnt/lustre2/lockdir/lockfile29967) error: No such file or directory total: 29966 unlinks in 95 seconds: 315.431580 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (23140) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (23143) - No such process PASS 12 (147s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 13: test directory page revocation ======= 20:09:01 (1712275741) f13 f13 PASS 13 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 20:09:04 (1712275744) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4661: 24742 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 20:09:07 (1712275747) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4661: 25480 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 20:09:10 (1712275750) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 26218 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 20:09:13 (1712275753) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 26957 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14d: chmod of executing file is still possible ================================================================== 20:09:16 (1712275756) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 27695 Terminated $DIR1/$tdir/sleep 60 PASS 14d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 15: test out-of-space with multiple writers ===================================================================== 20:09:19 (1712275759) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16a: 500 iterations of dual-mount fsx ==== 20:09:30 (1712275770) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.141737 s, 59.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.129734 s, 64.7 MB/s Chance of close/open is 1 in 50 Seed set to 5771 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1712275771.244842 write 0x3ec958 thru 0x3f3fd1 (0x767a bytes) 000002[0] 1712275771.255188 mapwrite 0x7c90b7 thru 0x7d50c0 (0xc00a bytes) 000003[1] 1712275771.297511 read 0x74e042 thru 0x74fff5 (0x1fb4 bytes) 000004[0] 1712275771.304324 mapwrite 0x2a8185 thru 0x2b3a13 (0xb88f bytes) 000005[0] 1712275771.331248 mapwrite 0x53b551 thru 0x53d2c9 (0x1d79 bytes) 000007[0] 1712275771.352229 write 0x108a90 thru 0x10e28e (0x57ff bytes) 000009[0] 1712275771.359742 mapwrite 0x590038 thru 0x596fa4 (0x6f6d bytes) 000010[1] 1712275771.381607 write 0x88e242 thru 0x895a75 (0x7834 bytes) 000011[0] 1712275771.391201 mapread 0x19405e thru 0x19f6a1 (0xb644 bytes) 000012[0] 1712275771.406220 read 0x7b36d5 thru 0x7b8e94 (0x57c0 bytes) 000014[0] 1712275771.412312 read 0x7e21cf thru 0x7e6dcd (0x4bff bytes) 000016[1] 1712275771.447918 read 0x5178ae thru 0x5233b5 (0xbb08 bytes) 000017[1] 1712275771.490674 read 0x378889 thru 0x37f057 (0x67cf bytes) 000018[0] 1712275771.491114 write 0x22cff9 thru 0x237e30 (0xae38 bytes) 000022[1] 1712275771.511237 write 0x75b1dd thru 0x7616e9 (0x650d bytes) 000023[1] 1712275771.522208 mapread 0x613e30 thru 0x61fbc0 (0xbd91 bytes) 000024[0] 1712275771.540547 write 0x087df2 thru 0x08966f (0x187e bytes) 000026[0] 1712275771.549194 read 0x00fde8 thru 0x0140d1 (0x42ea bytes) 000030[0] 1712275771.578429 mapwrite 0x92f2f5 thru 0x93a3fe (0xb10a bytes) 000033[0] 1712275771.645743 mapwrite 0x5442ca thru 0x551dd1 (0xdb08 bytes) 000035[1] 1712275771.665626 write 0x97be6a thru 0x98448d (0x8624 bytes) 000036[1] 1712275771.676733 read 0x113cb2 thru 0x122f9b (0xf2ea bytes) 000037[0] 1712275771.698069 mapwrite 0x5a7ca1 thru 0x5ace17 (0x5177 bytes) 000039[1] 1712275771.720345 write 0x7e96e6 thru 0x7f4eac (0xb7c7 bytes) 000041[0] 1712275771.749363 mapwrite 0x98b156 thru 0x997d2e (0xcbd9 bytes) 000042[0] 1712275771.790978 write 0x819262 thru 0x81c77b (0x351a bytes) truncating to largest ever: 0x949f4f 000043[0] 1712275771.793490 trunc from 0x997d2f to 0x949f4e (0x4dde0 bytes) 000044[0] 1712275771.797519 trunc from 0x949f4f to 0x30bfb9 (0x63df95 bytes) 000045[0] 1712275771.801826 mapwrite 0x58b8dd thru 0x58ed1e (0x3442 bytes) 000049[0] 1712275771.836613 write 0x331866 thru 0x337695 (0x5e30 bytes) 000050[1] 1712275771.842711 trunc from 0x58ed1f to 0x0d382b (0x4bb4f3 bytes) 000051[1] 1712275771.858269 mapwrite 0x9fcc25 thru 0x9fffff (0x33db bytes) 000053[0] 1712275771.899448 mapwrite 0x1098a5 thru 0x1140f3 (0xa84f bytes) 000055[1] 1712275771.949851 mapread 0x86c77a thru 0x879cc7 (0xd54e bytes) 000057[0] 1712275771.974833 mapread 0x239e50 thru 0x23eaa3 (0x4c54 bytes) 000059[1] 1712275771.991349 mapread 0x68a4b1 thru 0x68d110 (0x2c60 bytes) 000063[1] 1712275772.006079 read 0x4cd12a thru 0x4d847f (0xb356 bytes) 000065[0] 1712275772.041622 mapread 0x03916c thru 0x03b419 (0x22ae bytes) 000066[0] 1712275772.047111 mapread 0x7cf28d thru 0x7dbddc (0xcb50 bytes) 000068[1] 1712275772.070464 mapread 0x644bca thru 0x6480e8 (0x351f bytes) 000070[0] 1712275772.076850 trunc from 0xa00000 to 0x02baf2 (0x9d450d bytes) 000071[1] 1712275772.092614 read 0x019fb9 thru 0x01a1a1 (0x1e9 bytes) 000072[1] 1712275772.098579 trunc from 0x02baf3 to 0x54782d (0x51bd3b bytes) 000074[1] 1712275772.109035 write 0x3ffe89 thru 0x40b9bd (0xbb35 bytes) 000076[1] 1712275772.133331 trunc from 0x54782e to 0x67eb95 (0x137368 bytes) 000077[0] 1712275772.138092 mapwrite 0x0878a3 thru 0x095e1e (0xe57c bytes) 000080[0] 1712275772.192699 write 0x8f2e93 thru 0x9018e8 (0xea56 bytes) 000082[1] 1712275772.201577 read 0x4490a9 thru 0x44e215 (0x516d bytes) 000083[1] 1712275772.211631 mapwrite 0x126413 thru 0x128df5 (0x29e3 bytes) 000086[1] 1712275772.264653 read 0x2b41be thru 0x2c05a0 (0xc3e3 bytes) 000087[0] 1712275772.268300 read 0x58a32a thru 0x599cf2 (0xf9c9 bytes) 000089[0] 1712275772.295183 write 0x2828e4 thru 0x2878b2 (0x4fcf bytes) 000090[0] 1712275772.319234 trunc from 0x9018e9 to 0x7050ba (0x1fc82e bytes) 000091[1] 1712275772.327013 trunc from 0x7050bb to 0x15491c (0x5b079e bytes) 000092[1] 1712275772.342564 trunc from 0x15491d to 0x45a5d0 (0x305cb4 bytes) 000093[1] 1712275772.351183 mapwrite 0x6a1ba3 thru 0x6ac091 (0xa4ef bytes) 000096[1] 1712275772.401607 mapwrite 0x7a816a thru 0x7b01bc (0x8053 bytes) 000097[1] 1712275772.435025 trunc from 0x7b01bd to 0x3c0a48 (0x3ef774 bytes) 000099[0] 1712275772.449999 mapwrite 0x7c9b3e thru 0x7d6e25 (0xd2e8 bytes) 000105[0] 1712275772.514023 write 0x0787f2 thru 0x07beac (0x36bb bytes) 000107[1] 1712275772.520680 read 0x00bb2d thru 0x01a102 (0xe5d6 bytes) 000109[0] 1712275772.545743 read 0x02286f thru 0x023d67 (0x14f9 bytes) 000111[0] 1712275772.563211 read 0x59f11d thru 0x5ae054 (0xef38 bytes) 000112[0] 1712275772.568228 trunc from 0x7d6e26 to 0x2104ad (0x5c6978 bytes) 000115[0] 1712275772.592643 mapwrite 0x1dd7b0 thru 0x1de1ca (0xa1b bytes) 000116[1] 1712275772.623086 mapwrite 0x962a25 thru 0x96ba70 (0x904c bytes) 000117[0] 1712275772.678540 read 0x87d54d thru 0x87febf (0x2973 bytes) 000119[1] 1712275772.703952 mapread 0x2b9ba6 thru 0x2c8a41 (0xee9c bytes) 000121[0] 1712275772.727397 mapread 0x1d9984 thru 0x1dc13e (0x27bb bytes) 000123[1] 1712275772.732764 trunc from 0x96ba71 to 0x2c8e6d (0x6a2c03 bytes) 000125[0] 1712275772.750430 mapread 0x1ca55b thru 0x1ce524 (0x3fca bytes) 000126[1] 1712275772.763616 trunc from 0x2c8e6e to 0x8b81bc (0x5ef34f bytes) 000131[0] 1712275772.782300 mapread 0x85ea86 thru 0x8698a0 (0xae1b bytes) 000132[0] 1712275772.805239 trunc from 0x8b81bd to 0x65f441 (0x258d7b bytes) 000134[1] 1712275772.818769 mapread 0x619fe3 thru 0x625aaa (0xbac8 bytes) 000135[0] 1712275772.842987 write 0x5623bd thru 0x56f2a8 (0xceec bytes) 000139[1] 1712275772.866979 mapwrite 0x3ef389 thru 0x3f7f54 (0x8bcc bytes) 000141[1] 1712275772.907190 mapread 0x04fc7b thru 0x053bf2 (0x3f78 bytes) 000143[0] 1712275772.921755 write 0x0f9e9b thru 0x109e97 (0xfffd bytes) 000144[0] 1712275772.929729 write 0x348934 thru 0x34e6ee (0x5dbb bytes) 000146[1] 1712275772.938334 mapwrite 0x576b5c thru 0x58643d (0xf8e2 bytes) 000148[0] 1712275772.987517 write 0x736cbe thru 0x744290 (0xd5d3 bytes) 000149[0] 1712275772.993712 mapwrite 0x3ac17a thru 0x3b9324 (0xd1ab bytes) 000150[0] 1712275773.033476 read 0x260077 thru 0x26c31d (0xc2a7 bytes) 000151[0] 1712275773.062059 trunc from 0x744291 to 0x6bdb27 (0x86769 bytes) 000153[1] 1712275773.072219 read 0x50c178 thru 0x50fcee (0x3b77 bytes) 000154[0] 1712275773.082784 read 0x0843dd thru 0x085d40 (0x1964 bytes) 000156[0] 1712275773.089660 trunc from 0x6bdb28 to 0x484fc1 (0x238b66 bytes) 000158[0] 1712275773.105366 trunc from 0x484fc2 to 0x7d5d8e (0x350dcd bytes) 000164[1] 1712275773.126322 mapwrite 0x844273 thru 0x845f43 (0x1cd1 bytes) 000165[0] 1712275773.192273 trunc from 0x845f44 to 0x294f78 (0x5b0fcb bytes) 000170[0] 1712275773.224557 mapwrite 0x05cb53 thru 0x06713e (0xa5ec bytes) 000171[1] 1712275773.250793 read 0x10283c thru 0x106644 (0x3e09 bytes) 000173[0] 1712275773.269636 read 0x11274c thru 0x1223bd (0xfc72 bytes) 000174[1] 1712275773.286722 mapwrite 0x3749a2 thru 0x3756c5 (0xd24 bytes) 000177[0] 1712275773.326985 mapwrite 0x2472a4 thru 0x25142b (0xa188 bytes) 000178[0] 1712275773.371472 mapwrite 0x89a902 thru 0x89be28 (0x1527 bytes) 000179[0] 1712275773.416060 write 0x7f4dee thru 0x7f996d (0x4b80 bytes) 000181[1] 1712275773.417870 mapread 0x508880 thru 0x508b01 (0x282 bytes) 000182[1] 1712275773.431342 mapread 0x3799f3 thru 0x37fba0 (0x61ae bytes) 000183[1] 1712275773.441654 write 0x4d8247 thru 0x4e6c9a (0xea54 bytes) 000185[1] 1712275773.462463 write 0x480ee0 thru 0x48c2db (0xb3fc bytes) 000187[1] 1712275773.470340 read 0x646572 thru 0x6474e4 (0xf73 bytes) 000190[0] 1712275773.471050 trunc from 0x89be29 to 0x948371 (0xac549 bytes) 000194[1] 1712275773.494334 read 0x104844 thru 0x112b5c (0xe319 bytes) 000197[1] 1712275773.523795 read 0x3256d7 thru 0x3329ba (0xd2e4 bytes) 000202[1] 1712275773.531589 mapwrite 0x32abae thru 0x336c1b (0xc06e bytes) 000204[1] 1712275773.588562 trunc from 0x948372 to 0x4bdc17 (0x48a75a bytes) 000206[1] 1712275773.602334 trunc from 0x4bdc18 to 0x13e448 (0x37f7cf bytes) 000207[1] 1712275773.610211 read 0x13d0c0 thru 0x13d7b7 (0x6f8 bytes) 000209[1] 1712275773.610887 read 0x117726 thru 0x11c8f8 (0x51d3 bytes) 000221[0] 1712275773.633836 read 0x0c9b3c thru 0x0d1e6e (0x8333 bytes) 000222[1] 1712275773.649463 read 0x002095 thru 0x00eb2c (0xca98 bytes) 000223[1] 1712275773.661628 trunc from 0x13e449 to 0x5d7aae (0x499666 bytes) 000224[0] 1712275773.673081 mapwrite 0x16f226 thru 0x17439c (0x5177 bytes) 000227[0] 1712275773.723671 mapread 0x3de14d thru 0x3ea883 (0xc737 bytes) 000228[0] 1712275773.730649 mapwrite 0x9f9169 thru 0x9fffff (0x6e97 bytes) 000233[1] 1712275773.797825 mapwrite 0x2ebb5e thru 0x2ef857 (0x3cfa bytes) 000234[1] 1712275773.837877 mapwrite 0x5ba967 thru 0x5c2a8c (0x8126 bytes) 000235[1] 1712275773.875748 mapread 0x172833 thru 0x17dd6c (0xb53a bytes) 000239[1] 1712275773.889660 trunc from 0xa00000 to 0x2acba6 (0x753459 bytes) 000242[0] 1712275773.904852 read 0x0cb0ed thru 0x0cb49a (0x3ae bytes) 000243[1] 1712275773.920999 mapread 0x243556 thru 0x2464ea (0x2f95 bytes) 000244[1] 1712275773.926112 read 0x2677e8 thru 0x26eba5 (0x73be bytes) 000249[0] 1712275773.939351 trunc from 0x2acba7 to 0x560408 (0x2b3862 bytes) 000250[1] 1712275773.953126 write 0x3a48bf thru 0x3b3753 (0xee95 bytes) 000251[0] 1712275773.963066 write 0x4f2247 thru 0x4f7f78 (0x5d32 bytes) 000252[1] 1712275773.968903 trunc from 0x560409 to 0x607b8b (0xa7783 bytes) 000253[0] 1712275773.981675 trunc from 0x607b8c to 0x46789e (0x1a02ed bytes) 000257[0] 1712275774.000741 trunc from 0x46789f to 0x2b54e4 (0x1b23ba bytes) 000258[0] 1712275774.009589 mapwrite 0x4fca04 thru 0x507f07 (0xb504 bytes) 000260[0] 1712275774.046977 read 0x1fb5da thru 0x1fc123 (0xb4a bytes) 000262[1] 1712275774.064032 mapwrite 0x466533 thru 0x469432 (0x2f00 bytes) 000267[0] 1712275774.102676 mapwrite 0x9c4501 thru 0x9c8c46 (0x4746 bytes) 000269[1] 1712275774.153475 read 0x6b0f52 thru 0x6b931b (0x83ca bytes) 000275[1] 1712275774.180775 trunc from 0x9c8c47 to 0x0dc1f8 (0x8eca4e bytes) 000276[0] 1712275774.204820 mapread 0x04bcaa thru 0x04e907 (0x2c5e bytes) 000277[0] 1712275774.215138 trunc from 0x0dc1f9 to 0x582095 (0x4a5e9d bytes) 000278[1] 1712275774.226197 mapread 0x4c62d2 thru 0x4d43be (0xe0ed bytes) 000282[0] 1712275774.257731 read 0x37da14 thru 0x382fb2 (0x559f bytes) 000283[1] 1712275774.276972 mapwrite 0x8349ab thru 0x844501 (0xfb57 bytes) 000284[1] 1712275774.354958 mapread 0x4d1667 thru 0x4db8b7 (0xa251 bytes) 000285[1] 1712275774.355657 trunc from 0x844502 to 0x1e1143 (0x6633be bytes) 000287[1] 1712275774.372954 mapwrite 0x178d67 thru 0x17d765 (0x49ff bytes) 000288[0] 1712275774.391377 mapread 0x19890e thru 0x1a4c5e (0xc351 bytes) 000290[0] 1712275774.414371 write 0x2e9c37 thru 0x2f248a (0x8854 bytes) 000291[0] 1712275774.420666 mapwrite 0x4ce8bb thru 0x4dc02d (0xd773 bytes) 000292[1] 1712275774.459788 read 0x309c0f thru 0x315fa1 (0xc393 bytes) 000293[1] 1712275774.481998 read 0x2654ec thru 0x26b902 (0x6417 bytes) 000294[0] 1712275774.485438 write 0x81dae1 thru 0x822c9f (0x51bf bytes) 000295[0] 1712275774.495210 mapwrite 0x8a349a thru 0x8a7e88 (0x49ef bytes) 000297[0] 1712275774.542196 read 0x6b682f thru 0x6c5e35 (0xf607 bytes) 000298[0] 1712275774.545463 read 0x7fb3b7 thru 0x808ee6 (0xdb30 bytes) 000299[1] 1712275774.546782 write 0x04bcb0 thru 0x050cf4 (0x5045 bytes) 000301[0] 1712275774.568984 read 0x477028 thru 0x4796f0 (0x26c9 bytes) 000303[1] 1712275774.579040 read 0x7cd25a thru 0x7cd3cf (0x176 bytes) 000304[0] 1712275774.602427 mapread 0x80e296 thru 0x8193f9 (0xb164 bytes) 000308[0] 1712275774.649833 write 0x4949f9 thru 0x49f2bb (0xa8c3 bytes) 000309[1] 1712275774.659527 read 0x097924 thru 0x0a571f (0xddfc bytes) 000313[0] 1712275774.689047 mapwrite 0x171834 thru 0x1779fc (0x61c9 bytes) 000316[1] 1712275774.737343 mapread 0x3b2ef7 thru 0x3bd268 (0xa372 bytes) 000317[1] 1712275774.759615 mapwrite 0x3455cb thru 0x34b830 (0x6266 bytes) 000321[1] 1712275774.802776 mapread 0x5cb07f thru 0x5d5157 (0xa0d9 bytes) 000322[1] 1712275774.828858 write 0x83c042 thru 0x84a750 (0xe70f bytes) 000324[0] 1712275774.835512 read 0x5d619e thru 0x5db1e6 (0x5049 bytes) 000325[0] 1712275774.842965 mapread 0x39f103 thru 0x3ab2ca (0xc1c8 bytes) 000328[1] 1712275774.867574 read 0x36bb4b thru 0x3702f9 (0x47af bytes) 000330[1] 1712275774.886950 mapread 0x34fca5 thru 0x356860 (0x6bbc bytes) 000331[1] 1712275774.887483 mapwrite 0x35df34 thru 0x3621b7 (0x4284 bytes) 000335[0] 1712275774.936680 write 0x31e2be thru 0x329442 (0xb185 bytes) 000336[0] 1712275774.959058 write 0x4daf24 thru 0x4e2e88 (0x7f65 bytes) 000337[0] 1712275774.968374 mapwrite 0x6e74df thru 0x6f1604 (0xa126 bytes) 000341[1] 1712275775.008786 read 0x32a48d thru 0x333a52 (0x95c6 bytes) 000342[1] 1712275775.031974 write 0x9f68d3 thru 0x9fffff (0x972d bytes) 000343[0] 1712275775.042596 trunc from 0xa00000 to 0x776447 (0x289bb8 bytes) 000344[1] 1712275775.052974 trunc from 0x776448 to 0x5d00ab (0x1a639c bytes) 000346[1] 1712275775.067448 read 0x15c80c thru 0x16ab82 (0xe377 bytes) 000349[0] 1712275775.095234 read 0x586ad0 thru 0x594f4f (0xe480 bytes) 000354[0] 1712275775.121801 read 0x1cd0b1 thru 0x1d6e77 (0x9dc7 bytes) 000355[0] 1712275775.148711 trunc from 0x5d00ac to 0x2bd932 (0x312779 bytes) 000356[0] 1712275775.172308 mapread 0x0beaab thru 0x0c321e (0x4774 bytes) 000357[1] 1712275775.181954 read 0x154f3c thru 0x15cb84 (0x7c49 bytes) 000362[1] 1712275775.216770 trunc from 0x2bd933 to 0x084676 (0x2392bc bytes) 000363[0] 1712275775.234117 read 0x045a39 thru 0x04cb4e (0x7116 bytes) 000364[1] 1712275775.243173 read 0x028ad4 thru 0x02ab50 (0x207d bytes) 000365[1] 1712275775.251803 write 0x9eaa89 thru 0x9f174e (0x6cc6 bytes) 000368[0] 1712275775.266233 mapwrite 0x4a9e82 thru 0x4ade45 (0x3fc4 bytes) 000370[1] 1712275775.316325 mapread 0x93bcbe thru 0x949477 (0xd7ba bytes) 000371[1] 1712275775.347391 write 0x70ecd2 thru 0x7110ef (0x241e bytes) 000378[0] 1712275775.365664 read 0x1c8836 thru 0x1cd933 (0x50fe bytes) 000381[0] 1712275775.402481 mapread 0x85d65d thru 0x8695d6 (0xbf7a bytes) 000382[0] 1712275775.426922 read 0x0bfedd thru 0x0c34ab (0x35cf bytes) 000383[1] 1712275775.431382 mapread 0x186d8b thru 0x18e96d (0x7be3 bytes) 000386[1] 1712275775.455022 read 0x5d4b96 thru 0x5e2d7c (0xe1e7 bytes) 000387[1] 1712275775.459039 read 0x666ccc thru 0x66fa26 (0x8d5b bytes) 000389[1] 1712275775.462972 trunc from 0x9f174f to 0x65b34c (0x396402 bytes) 000397[1] 1712275775.493343 trunc from 0x65b34d to 0x19056b (0x4cade1 bytes) 000403[0] 1712275775.520167 mapread 0x114d58 thru 0x122819 (0xdac2 bytes) 000404[0] 1712275775.535478 mapwrite 0x4927c8 thru 0x49d7cc (0xb005 bytes) 000406[1] 1712275775.580203 mapwrite 0x5300ba thru 0x53d596 (0xd4dd bytes) 000407[1] 1712275775.635149 read 0x2b6c2e thru 0x2bff99 (0x936c bytes) 000408[1] 1712275775.638398 mapread 0x3caf10 thru 0x3cbf1e (0x100f bytes) 000409[1] 1712275775.642130 mapwrite 0x1c92a0 thru 0x1ce1a9 (0x4f0a bytes) 000410[1] 1712275775.663333 mapwrite 0x191252 thru 0x192fed (0x1d9c bytes) 000415[0] 1712275775.687882 read 0x32bde3 thru 0x33b091 (0xf2af bytes) 000416[0] 1712275775.712238 write 0x53f767 thru 0x54a326 (0xabc0 bytes) 000417[0] 1712275775.716236 write 0x7a20f7 thru 0x7a47c2 (0x26cc bytes) 000418[1] 1712275775.717046 mapread 0x59023d thru 0x595991 (0x5755 bytes) 000419[1] 1712275775.748839 trunc from 0x7a47c3 to 0x7d50ff (0x3093d bytes) 000420[1] 1712275775.765719 mapwrite 0x29530b thru 0x2a0ee9 (0xbbdf bytes) 000421[0] 1712275775.810756 mapread 0x2c4304 thru 0x2d0d44 (0xca41 bytes) 000423[0] 1712275775.829836 trunc from 0x7d5100 to 0x433cbc (0x3a1443 bytes) 000425[1] 1712275775.843970 read 0x0e23e8 thru 0x0ecd24 (0xa93d bytes) 000426[0] 1712275775.864445 mapread 0x35cedc thru 0x365a97 (0x8bbc bytes) 000427[1] 1712275775.881113 mapwrite 0x5ed67d thru 0x5f1f33 (0x48b7 bytes) 000434[0] 1712275775.938305 write 0x0e2d84 thru 0x0e49f8 (0x1c75 bytes) 000435[1] 1712275775.946095 read 0x0e2400 thru 0x0ef7f7 (0xd3f8 bytes) 000436[0] 1712275775.966014 trunc from 0x5f1f34 to 0x757905 (0x1659d2 bytes) 000438[1] 1712275775.981144 write 0x853c8c thru 0x85f499 (0xb80e bytes) 000439[1] 1712275775.988596 write 0x5f5225 thru 0x5fc970 (0x774c bytes) 000440[0] 1712275776.001098 read 0x6cd9f9 thru 0x6dd3d6 (0xf9de bytes) 000442[0] 1712275776.026372 mapwrite 0x1c10be thru 0x1ca413 (0x9356 bytes) 000448[0] 1712275776.090637 mapwrite 0x7bd4c0 thru 0x7c80c3 (0xac04 bytes) 000449[0] 1712275776.134970 trunc from 0x85f49a to 0x71f709 (0x13fd90 bytes) 000451[0] 1712275776.144909 read 0x60f842 thru 0x612d8a (0x3549 bytes) 000452[0] 1712275776.148789 mapwrite 0x911c5b thru 0x91d58f (0xb935 bytes) 000455[1] 1712275776.197580 mapread 0x2f1d2a thru 0x2facbf (0x8f96 bytes) 000457[0] 1712275776.223333 mapwrite 0x4956d9 thru 0x497769 (0x2091 bytes) 000458[0] 1712275776.236632 read 0x48b472 thru 0x497b58 (0xc6e7 bytes) 000459[1] 1712275776.236939 mapread 0x0f00c4 thru 0x0f776d (0x76aa bytes) 000460[0] 1712275776.251179 mapwrite 0x2c0d0e thru 0x2c10ab (0x39e bytes) 000461[1] 1712275776.294344 read 0x77803b thru 0x77dcf7 (0x5cbd bytes) 000467[1] 1712275776.322168 write 0x1a4d2a thru 0x1aba40 (0x6d17 bytes) 000470[0] 1712275776.357907 mapread 0x16d154 thru 0x17bb31 (0xe9de bytes) 000471[0] 1712275776.404863 write 0x7adc8c thru 0x7af16d (0x14e2 bytes) truncating to largest ever: 0x96cb91 000472[1] 1712275776.413827 trunc from 0x91d590 to 0x96cb90 (0x4f601 bytes) 000473[1] 1712275776.429941 read 0x57de42 thru 0x57e50f (0x6ce bytes) 000474[0] 1712275776.467043 read 0x73753d thru 0x73f527 (0x7feb bytes) 000478[1] 1712275776.491995 read 0x5fe3df thru 0x60a4c7 (0xc0e9 bytes) 000480[0] 1712275776.515489 mapread 0x4cd7aa thru 0x4db238 (0xda8f bytes) 000482[0] 1712275776.540697 mapwrite 0x8308c3 thru 0x84051e (0xfc5c bytes) 000484[0] 1712275776.611247 write 0x6c7559 thru 0x6cc782 (0x522a bytes) 000486[1] 1712275776.625837 mapread 0x8337b2 thru 0x839124 (0x5973 bytes) 000491[1] 1712275776.649734 trunc from 0x96cb91 to 0x7103aa (0x25c7e6 bytes) 000494[0] 1712275776.667332 read 0x382891 thru 0x38abf2 (0x8362 bytes) 000495[0] 1712275776.690010 mapread 0x0a481f thru 0x0b3f58 (0xf73a bytes) 000498[0] 1712275776.696708 mapread 0x6a83ac thru 0x6b0fa3 (0x8bf8 bytes) 000499[1] 1712275776.715988 trunc from 0x7103ab to 0x8c8d8d (0x1b89e3 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5776 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1712275776.963287 write 0x49f000 thru 0x4ae3ff (0xf400 bytes) 000002[1] 1712275776.973110 write_OD 0x24e000 thru 0x253fff (0x6000 bytes) truncating to largest ever: 0x746c96 000005[1] 1712275776.994225 trunc from 0x4ae400 to 0x746c95 (0x298896 bytes) 000007[1] 1712275777.008947 read_OD 0x166000 thru 0x16bfff (0x6000 bytes) 000009[0] 1712275777.014594 read 0x16e000 thru 0x177128 (0x9129 bytes) 000010[1] 1712275777.017747 mapwrite 0x6dc000 thru 0x6e6819 (0xa81a bytes) 000012[1] 1712275777.054107 trunc from 0x746c96 to 0x1f266a (0x55462b bytes) 000013[1] 1712275777.058795 write_OD 0x253000 thru 0x255fff (0x3000 bytes) 000015[1] 1712275777.075778 write_OD 0x57e000 thru 0x57efff (0x1000 bytes) 000016[0] 1712275777.092063 mapread 0x0de000 thru 0x0e0b1a (0x2b1b bytes) 000020[1] 1712275777.098552 mapwrite 0x5b2000 thru 0x5b2c6f (0xc70 bytes) 000021[1] 1712275777.122097 mapread 0x0d7000 thru 0x0dc303 (0x5304 bytes) 000023[1] 1712275777.132303 mapread 0x2c1000 thru 0x2c39a9 (0x29aa bytes) 000025[0] 1712275777.140513 mapwrite 0x87c000 thru 0x87c59a (0x59b bytes) 000028[0] 1712275777.170481 read 0x345000 thru 0x3481a9 (0x31aa bytes) 000029[0] 1712275777.203484 mapread 0x1b5000 thru 0x1b945e (0x445f bytes) 000030[1] 1712275777.207064 write_OD 0x2b2000 thru 0x2b4fff (0x3000 bytes) 000033[0] 1712275777.235024 write 0x904000 thru 0x90e1d3 (0xa1d4 bytes) 000034[1] 1712275777.242313 mapread 0x770000 thru 0x77e135 (0xe136 bytes) 000035[1] 1712275777.256277 mapwrite 0x93b000 thru 0x9468da (0xb8db bytes) 000037[0] 1712275777.296736 mapread 0x4f4000 thru 0x4f86b5 (0x46b6 bytes) truncating to largest ever: 0x8af73d 000039[1] 1712275777.313681 trunc from 0x9468db to 0x8af73c (0x9719e bytes) 000040[0] 1712275777.321236 mapwrite 0x90e000 thru 0x91470e (0x670f bytes) 000046[1] 1712275777.362925 mapread 0x84e000 thru 0x84ea5f (0xa60 bytes) 000047[0] 1712275777.369131 read 0x8e6000 thru 0x8f39bb (0xd9bc bytes) 000048[1] 1712275777.411690 read_OD 0x183000 thru 0x185fff (0x3000 bytes) 000049[1] 1712275777.413679 mapread 0x905000 thru 0x905a1a (0xa1b bytes) 000050[0] 1712275777.446393 trunc from 0x91470f to 0x06f1db (0x8a5533 bytes) 000051[1] 1712275777.469692 read_OD 00000000 thru 0x001fff (0x2000 bytes) 000052[0] 1712275777.475711 read 0x066000 thru 0x06e492 (0x8493 bytes) 000056[0] 1712275777.483324 mapwrite 0x1ac000 thru 0x1ac6fd (0x6fe bytes) 000058[1] 1712275777.521664 trunc from 0x1ac6fe to 0x485c0a (0x2d950d bytes) 000061[0] 1712275777.529911 write 0x8dc000 thru 0x8e6c9e (0xac9f bytes) 000062[0] 1712275777.535020 mapwrite 0x4b4000 thru 0x4bd8da (0x98db bytes) 000063[0] 1712275777.574783 write 0x71a000 thru 0x71f2b9 (0x52ba bytes) 000066[0] 1712275777.582620 trunc from 0x8e6c9f to 0x0209cc (0x8c62d2 bytes) 000069[0] 1712275777.598023 read 0x010000 thru 0x014c07 (0x4c08 bytes) 000071[1] 1712275777.601188 mapwrite 0x126000 thru 0x127b0e (0x1b0f bytes) 000076[0] 1712275777.645060 trunc from 0x127b0f to 0x5dfd0d (0x4b81ff bytes) 000077[1] 1712275777.655496 mapwrite 0x429000 thru 0x434f31 (0xbf32 bytes) 000082[1] 1712275777.709315 read_OD 0x156000 thru 0x160fff (0xb000 bytes) 000085[1] 1712275777.720045 mapwrite 0x1c4000 thru 0x1cc21b (0x821c bytes) 000087[0] 1712275777.763425 mapread 0x084000 thru 0x08ed46 (0xad47 bytes) 000090[1] 1712275777.792489 mapread 0x01f000 thru 0x0264cf (0x74d0 bytes) 000092[0] 1712275777.801453 trunc from 0x5dfd0e to 0x654759 (0x74a4c bytes) 000094[0] 1712275777.820915 write 0x3c6000 thru 0x3cd7cb (0x77cc bytes) 000097[0] 1712275777.835250 trunc from 0x65475a to 0x361c23 (0x2f2b36 bytes) 000098[1] 1712275777.841328 read_OD 0x0b7000 thru 0x0c0fff (0xa000 bytes) 000102[1] 1712275777.851823 trunc from 0x361c24 to 0x75dcd5 (0x3fc0b2 bytes) 000103[1] 1712275777.860428 read_OD 0x666000 thru 0x66efff (0x9000 bytes) 000107[0] 1712275777.867276 trunc from 0x75dcd6 to 0x37e215 (0x3dfac0 bytes) 000108[1] 1712275777.873305 mapwrite 0x401000 thru 0x4044a6 (0x34a7 bytes) 000109[0] 1712275777.916058 mapread 0x09e000 thru 0x0a0750 (0x2751 bytes) 000110[1] 1712275777.928240 mapread 0x0e5000 thru 0x0e6386 (0x1387 bytes) truncating to largest ever: 0x92290a 000112[0] 1712275777.932849 trunc from 0x4044a7 to 0x922909 (0x51e463 bytes) 000113[1] 1712275777.941038 trunc from 0x92290a to 0x8dd84b (0x450be bytes) 000119[0] 1712275777.954258 read 0x2c0000 thru 0x2ce1f0 (0xe1f1 bytes) truncating to largest ever: 0x9708eb 000120[1] 1712275777.987864 trunc from 0x8dd84c to 0x9708ea (0x9309f bytes) 000121[0] 1712275778.002378 mapread 0x7a3000 thru 0x7a301b (0x1c bytes) 000123[0] 1712275778.014833 read 0x924000 thru 0x92b6e9 (0x76ea bytes) 000124[0] 1712275778.015076 trunc from 0x9708eb to 0x40a446 (0x5664a4 bytes) 000125[1] 1712275778.021857 read_OD 0x2e0000 thru 0x2ecfff (0xd000 bytes) 000128[0] 1712275778.029320 mapwrite 0x21d000 thru 0x21e154 (0x1155 bytes) 000130[1] 1712275778.069211 mapwrite 0x26f000 thru 0x2724e8 (0x34e9 bytes) 000133[1] 1712275778.125229 read_OD 0x1f5000 thru 0x1fcfff (0x8000 bytes) 000138[0] 1712275778.135074 read 0x0a5000 thru 0x0b03f2 (0xb3f3 bytes) 000139[0] 1712275778.150315 trunc from 0x40a447 to 0x8feee2 (0x4f4a9c bytes) 000140[0] 1712275778.165031 trunc from 0x8feee3 to 0x2c52e4 (0x639bfe bytes) 000143[0] 1712275778.175295 read 0x129000 thru 0x12d4f3 (0x44f4 bytes) 000144[0] 1712275778.190167 read 0x2a7000 thru 0x2aa0c3 (0x30c4 bytes) 000146[1] 1712275778.190680 mapwrite 0x27f000 thru 0x282875 (0x3876 bytes) 000148[1] 1712275778.227509 trunc from 0x2c52e5 to 0x65e154 (0x398e70 bytes) 000149[1] 1712275778.234105 mapwrite 0x3db000 thru 0x3dc8fc (0x18fd bytes) 000150[1] 1712275778.268306 trunc from 0x65e155 to 0x20f54a (0x44ec0a bytes) 000152[1] 1712275778.280982 trunc from 0x20f54b to 0x12c84f (0xe2cfb bytes) 000155[1] 1712275778.291283 mapread 0x09b000 thru 0x09bd73 (0xd74 bytes) 000156[0] 1712275778.291773 write 0x36b000 thru 0x378125 (0xd126 bytes) 000157[1] 1712275778.296630 read_OD 0x0b5000 thru 0x0b9fff (0x5000 bytes) 000158[1] 1712275778.304757 trunc from 0x378126 to 0x3ab22b (0x33106 bytes) 000162[0] 1712275778.322027 mapwrite 0x225000 thru 0x230bab (0xbbac bytes) 000163[0] 1712275778.361813 mapread 0x2be000 thru 0x2bf543 (0x1544 bytes) 000165[0] 1712275778.365613 trunc from 0x3ab22c to 0x352f27 (0x58304 bytes) 000168[0] 1712275778.375784 read 0x122000 thru 0x12b349 (0x934a bytes) 000169[1] 1712275778.379283 mapread 0x101000 thru 0x103d15 (0x2d16 bytes) 000171[0] 1712275778.402229 write 0x4ea000 thru 0x4f366e (0x966f bytes) 000172[1] 1712275778.410120 mapwrite 0x3b6000 thru 0x3c0f0e (0xaf0f bytes) 000174[0] 1712275778.448872 mapwrite 0x13a000 thru 0x14555b (0xb55c bytes) 000182[1] 1712275778.529287 mapwrite 0x5c9000 thru 0x5d45e8 (0xb5e9 bytes) 000183[1] 1712275778.566385 trunc from 0x5d45e9 to 0x564d0b (0x6f8dd bytes) 000184[1] 1712275778.574199 trunc from 0x564d0c to 0x6f26fd (0x18d9f2 bytes) 000187[0] 1712275778.582924 write 0x9fa000 thru 0x9ff021 (0x5022 bytes) 000188[1] 1712275778.589685 trunc from 0x9ff022 to 0x4be2b2 (0x540d6f bytes) 000189[0] 1712275778.597916 trunc from 0x4be2b3 to 0x581d16 (0xc3a64 bytes) 000190[0] 1712275778.607791 trunc from 0x581d17 to 0x423744 (0x15e5d2 bytes) 000191[1] 1712275778.612464 trunc from 0x423745 to 0x4260cf (0x298b bytes) 000192[0] 1712275778.622086 write 0x06e000 thru 0x06f906 (0x1907 bytes) 000194[0] 1712275778.626069 mapwrite 0x7b0000 thru 0x7b1346 (0x1347 bytes) 000197[0] 1712275778.672346 mapwrite 0x812000 thru 0x8129f7 (0x9f8 bytes) 000199[1] 1712275778.704044 mapwrite 0x7bd000 thru 0x7c320f (0x6210 bytes) 000201[0] 1712275778.744283 write 0x1fa000 thru 0x1ffa95 (0x5a96 bytes) 000202[0] 1712275778.754562 mapread 0x387000 thru 0x389d10 (0x2d11 bytes) 000205[1] 1712275778.761970 trunc from 0x8129f8 to 0x77f454 (0x935a3 bytes) 000206[0] 1712275778.774231 write 0x4a9000 thru 0x4b0e22 (0x7e23 bytes) 000211[0] 1712275778.795968 trunc from 0x77f455 to 0x56be40 (0x213614 bytes) 000214[1] 1712275778.808403 mapwrite 0x3e2000 thru 0x3e2fdf (0xfe0 bytes) 000218[0] 1712275778.838040 write 0x488000 thru 0x497a25 (0xfa26 bytes) 000219[0] 1712275778.845001 mapwrite 0x5e4000 thru 0x5e9436 (0x5437 bytes) 000223[0] 1712275778.879613 read 0x147000 thru 0x156f96 (0xff97 bytes) 000224[1] 1712275778.906086 read_OD 0x064000 thru 0x065fff (0x2000 bytes) 000227[1] 1712275778.917068 mapread 0x315000 thru 0x31592c (0x92d bytes) 000228[0] 1712275778.939866 read 0x52d000 thru 0x535733 (0x8734 bytes) 000233[1] 1712275778.962339 mapwrite 0x42c000 thru 0x439201 (0xd202 bytes) 000234[1] 1712275779.014948 trunc from 0x5e9437 to 0x07b5e3 (0x56de53 bytes) 000235[1] 1712275779.024172 read_OD 0x01f000 thru 0x026fff (0x8000 bytes) 000236[0] 1712275779.029347 write 0x951000 thru 0x956173 (0x5174 bytes) 000237[0] 1712275779.033688 read 0x878000 thru 0x87e40e (0x640f bytes) 000244[0] 1712275779.063271 mapwrite 0x160000 thru 0x16c856 (0xc857 bytes) 000245[0] 1712275779.081769 mapwrite 0x2c9000 thru 0x2d28fb (0x98fc bytes) 000247[0] 1712275779.098762 write 0x7e9000 thru 0x7f1e56 (0x8e57 bytes) 000250[0] 1712275779.102686 read 0x038000 thru 0x038149 (0x14a bytes) 000252[1] 1712275779.103019 read_OD 0x738000 thru 0x73efff (0x7000 bytes) truncating to largest ever: 0x9a9e18 000254[1] 1712275779.120363 trunc from 0x956174 to 0x9a9e17 (0x53ca4 bytes) 000256[1] 1712275779.133214 mapwrite 0x8e2000 thru 0x8ee3b8 (0xc3b9 bytes) 000257[0] 1712275779.181402 write 0x4d8000 thru 0x4e53a4 (0xd3a5 bytes) 000259[0] 1712275779.216607 read 0x5c6000 thru 0x5c92c1 (0x32c2 bytes) 000261[0] 1712275779.244978 trunc from 0x9a9e18 to 0x113904 (0x896513 bytes) 000262[1] 1712275779.257749 read_OD 0x0d9000 thru 0x0dafff (0x2000 bytes) 000264[0] 1712275779.264527 mapread 0x103000 thru 0x10e529 (0xb52a bytes) 000265[1] 1712275779.277593 mapread 0x0cd000 thru 0x0d2530 (0x5531 bytes) 000266[0] 1712275779.286060 read 0x0e5000 thru 0x0e7fe0 (0x2fe1 bytes) 000267[0] 1712275779.292669 write 0x18d000 thru 0x18f50b (0x250c bytes) 000269[0] 1712275779.302784 trunc from 0x18f50c to 0x78be24 (0x5fc919 bytes) 000270[1] 1712275779.311894 write_OD 0x8ae000 thru 0x8bbfff (0xe000 bytes) 000272[0] 1712275779.330122 mapwrite 0x517000 thru 0x523735 (0xc736 bytes) 000273[0] 1712275779.379825 read 0x2d2000 thru 0x2d7388 (0x5389 bytes) 000275[0] 1712275779.385989 write 0x88e000 thru 0x895474 (0x7475 bytes) 000276[1] 1712275779.389644 mapwrite 0x55d000 thru 0x568405 (0xb406 bytes) 000277[1] 1712275779.447879 write_OD 0x1b8000 thru 0x1b8fff (0x1000 bytes) 000279[0] 1712275779.473944 mapwrite 0x34e000 thru 0x35a75e (0xc75f bytes) 000281[1] 1712275779.527848 mapread 0x503000 thru 0x5093ae (0x63af bytes) 000282[1] 1712275779.546514 read_OD 0x006000 thru 0x00cfff (0x7000 bytes) 000289[1] 1712275779.556617 write_OD 0x632000 thru 0x640fff (0xf000 bytes) 000290[1] 1712275779.571224 mapwrite 0x8dd000 thru 0x8dfbb7 (0x2bb8 bytes) 000291[0] 1712275779.635187 read_OD 0x4d2000 thru 0x4d2fff (0x1000 bytes) 000292[0] 1712275779.648154 write_OD 0x464000 thru 0x471fff (0xe000 bytes) 000293[1] 1712275779.664164 read_OD 0x738000 thru 0x739fff (0x2000 bytes) 000294[0] 1712275779.666683 read_OD 0x6cf000 thru 0x6d3fff (0x5000 bytes) 000295[1] 1712275779.668850 write_OD 0x8da000 thru 0x8e8fff (0xf000 bytes) 000296[0] 1712275779.684038 mapwrite 0x910000 thru 0x919c44 (0x9c45 bytes) 000298[0] 1712275779.748289 trunc from 0x919c45 to 0x5abf75 (0x36dccf bytes) 000299[0] 1712275779.755818 write_OD 0x570000 thru 0x575fff (0x6000 bytes) 000300[0] 1712275779.775818 write_OD 0x755000 thru 0x75efff (0xa000 bytes) 000301[1] 1712275779.793356 mapwrite 0x1f2000 thru 0x1f9b06 (0x7b07 bytes) 000303[1] 1712275779.827240 trunc from 0x75f000 to 0x08f552 (0x6cfaad bytes) 000304[1] 1712275779.839952 read_OD 0x05f000 thru 0x06bfff (0xd000 bytes) 000305[0] 1712275779.845847 trunc from 0x08f553 to 0x505563 (0x476011 bytes) 000306[0] 1712275779.856198 mapread 0x4e4000 thru 0x4ee467 (0xa468 bytes) 000308[0] 1712275779.874563 mapread 0x47a000 thru 0x47ddd6 (0x3dd7 bytes) 000309[1] 1712275779.880551 read_OD 0x4c9000 thru 0x4cffff (0x7000 bytes) 000311[1] 1712275779.886805 mapwrite 0x005000 thru 0x009b83 (0x4b84 bytes) 000314[0] 1712275779.933081 write_OD 0x5e7000 thru 0x5effff (0x9000 bytes) 000315[1] 1712275779.956821 read_OD 0x1f5000 thru 0x202fff (0xe000 bytes) 000316[1] 1712275779.960994 mapwrite 0x373000 thru 0x37b697 (0x8698 bytes) 000322[0] 1712275780.020562 read_OD 0x539000 thru 0x539fff (0x1000 bytes) 000324[0] 1712275780.032002 mapwrite 0x1b1000 thru 0x1bb62f (0xa630 bytes) 000326[1] 1712275780.077178 mapwrite 0x2f2000 thru 0x2ffa83 (0xda84 bytes) 000329[0] 1712275780.121954 write_OD 0x57b000 thru 0x583fff (0x9000 bytes) 000331[0] 1712275780.138233 read_OD 0x207000 thru 0x213fff (0xd000 bytes) 000336[1] 1712275780.140894 trunc from 0x5f0000 to 0x83fa83 (0x24fa84 bytes) 000337[0] 1712275780.151116 write_OD 0x874000 thru 0x881fff (0xe000 bytes) 000338[0] 1712275780.167064 write_OD 0x1c8000 thru 0x1cafff (0x3000 bytes) 000339[1] 1712275780.184701 mapread 0x221000 thru 0x228a6a (0x7a6b bytes) 000341[0] 1712275780.206467 mapwrite 0x021000 thru 0x02aa3b (0x9a3c bytes) 000344[0] 1712275780.256989 read_OD 0x7d0000 thru 0x7d7fff (0x8000 bytes) 000345[1] 1712275780.268316 mapwrite 0x9fd000 thru 0x9ffe4e (0x2e4f bytes) 000347[1] 1712275780.305012 write_OD 0x78d000 thru 0x792fff (0x6000 bytes) 000352[0] 1712275780.323737 write_OD 0x1a1000 thru 0x1a9fff (0x9000 bytes) 000355[1] 1712275780.340162 mapwrite 0x2b4000 thru 0x2ba913 (0x6914 bytes) 000356[0] 1712275780.366163 mapwrite 0x56c000 thru 0x5706c3 (0x46c4 bytes) 000358[0] 1712275780.426033 trunc from 0x9ffe4f to 0x488ba1 (0x5772ad bytes) 000363[1] 1712275780.441820 write_OD 0x840000 thru 0x843fff (0x4000 bytes) 000364[0] 1712275780.464479 write_OD 0x0e2000 thru 0x0f0fff (0xf000 bytes) 000366[1] 1712275780.485738 mapread 0x6de000 thru 0x6ecd39 (0xed3a bytes) 000367[1] 1712275780.502652 write_OD 0x664000 thru 0x66afff (0x7000 bytes) 000368[0] 1712275780.519215 mapwrite 0x2f3000 thru 0x2f71ce (0x41cf bytes) 000370[1] 1712275780.552277 read_OD 0x475000 thru 0x483fff (0xf000 bytes) 000371[0] 1712275780.560022 trunc from 0x844000 to 0x1395a8 (0x70aa57 bytes) 000372[1] 1712275780.571726 mapwrite 0x379000 thru 0x3829b5 (0x99b6 bytes) 000376[1] 1712275780.615108 mapwrite 0x24c000 thru 0x253e0d (0x7e0e bytes) 000377[1] 1712275780.631985 mapread 0x011000 thru 0x01d930 (0xc931 bytes) 000379[0] 1712275780.636427 trunc from 0x3829b6 to 0x246da4 (0x13bc11 bytes) 000381[0] 1712275780.650022 mapread 0x0c3000 thru 0x0d0cdb (0xdcdc bytes) 000382[1] 1712275780.669375 read_OD 0x0aa000 thru 0x0b5fff (0xc000 bytes) 000385[1] 1712275780.676747 read_OD 0x20d000 thru 0x215fff (0x9000 bytes) 000388[0] 1712275780.679944 read_OD 0x0de000 thru 0x0e6fff (0x9000 bytes) 000389[1] 1712275780.683686 write_OD 0x34f000 thru 0x352fff (0x4000 bytes) 000390[1] 1712275780.700886 read_OD 0x03c000 thru 0x049fff (0xe000 bytes) 000393[0] 1712275780.703661 mapread 0x231000 thru 0x2378cd (0x68ce bytes) 000395[1] 1712275780.719427 mapwrite 0x0c2000 thru 0x0cf155 (0xd156 bytes) 000397[1] 1712275780.756204 trunc from 0x353000 to 0x1833f7 (0x1cfc08 bytes) 000402[0] 1712275780.771597 read_OD 0x06e000 thru 0x07afff (0xd000 bytes) 000403[0] 1712275780.779635 read_OD 0x0e8000 thru 0x0f4fff (0xd000 bytes) 000405[0] 1712275780.781863 read_OD 0x151000 thru 0x15afff (0xa000 bytes) 000407[0] 1712275780.784984 read_OD 0x007000 thru 0x010fff (0xa000 bytes) 000408[1] 1712275780.786956 trunc from 0x1833f8 to 0x0784f9 (0x10aefe bytes) 000410[1] 1712275780.798101 trunc from 0x0784fa to 0x6c92e2 (0x650de9 bytes) 000412[1] 1712275780.804911 write_OD 0x2bd000 thru 0x2cbfff (0xf000 bytes) 000413[1] 1712275780.821641 mapread 0x41d000 thru 0x42c515 (0xf516 bytes) truncating to largest ever: 0x9b7870 000419[0] 1712275780.845340 trunc from 0x6c92e3 to 0x9b786f (0x2ee58d bytes) 000420[1] 1712275780.860640 write_OD 0x3f2000 thru 0x3f4fff (0x3000 bytes) 000421[1] 1712275780.880379 trunc from 0x9b7870 to 0x716bfa (0x2a0c75 bytes) 000422[1] 1712275780.889728 trunc from 0x716bfb to 0x64c966 (0xca294 bytes) 000423[1] 1712275780.898391 mapread 0x0db000 thru 0x0dce13 (0x1e14 bytes) 000426[0] 1712275780.906659 mapread 0x0c7000 thru 0x0cee41 (0x7e42 bytes) 000428[0] 1712275780.937573 trunc from 0x64c967 to 0x64c550 (0x416 bytes) 000430[1] 1712275780.958206 read_OD 0x1e8000 thru 0x1eefff (0x7000 bytes) 000431[0] 1712275780.966263 trunc from 0x64c551 to 0x30f5f8 (0x33cf58 bytes) 000439[0] 1712275780.993331 trunc from 0x30f5f9 to 0x8c57ce (0x5b61d6 bytes) 000440[1] 1712275781.006043 read_OD 0x26d000 thru 0x27afff (0xe000 bytes) 000442[1] 1712275781.016247 mapwrite 0x305000 thru 0x313252 (0xe253 bytes) 000443[1] 1712275781.076547 write_OD 0x344000 thru 0x351fff (0xe000 bytes) 000444[1] 1712275781.101491 mapwrite 0x042000 thru 0x050d70 (0xed71 bytes) 000447[0] 1712275781.145801 trunc from 0x8c57cf to 0x066acd (0x85ed01 bytes) 000450[0] 1712275781.157925 read_OD 0x020000 thru 0x026fff (0x7000 bytes) 000451[0] 1712275781.161969 read_OD 0x05d000 thru 0x05efff (0x2000 bytes) 000455[1] 1712275781.164600 mapwrite 0x03a000 thru 0x03b103 (0x1104 bytes) 000456[0] 1712275781.189014 mapwrite 0x597000 thru 0x59f52c (0x852d bytes) 000457[1] 1712275781.245162 write_OD 0x544000 thru 0x546fff (0x3000 bytes) 000458[0] 1712275781.272152 mapread 0x556000 thru 0x55dbd0 (0x7bd1 bytes) 000460[0] 1712275781.288453 read_OD 0x23a000 thru 0x248fff (0xf000 bytes) 000461[0] 1712275781.291580 mapread 0x4a8000 thru 0x4ac21b (0x421c bytes) 000462[0] 1712275781.319005 mapread 0x41e000 thru 0x42dd89 (0xfd8a bytes) 000463[1] 1712275781.319403 mapwrite 0x0b5000 thru 0x0b65ec (0x15ed bytes) 000464[0] 1712275781.348252 read_OD 0x44b000 thru 0x44cfff (0x2000 bytes) 000468[1] 1712275781.355091 mapwrite 0x679000 thru 0x681f47 (0x8f48 bytes) 000469[1] 1712275781.395205 mapwrite 0x330000 thru 0x33c15e (0xc15f bytes) 000471[0] 1712275781.438421 read_OD 0x57b000 thru 0x57bfff (0x1000 bytes) 000474[0] 1712275781.445983 mapread 0x3ac000 thru 0x3b99f4 (0xd9f5 bytes) 000475[1] 1712275781.474239 write_OD 0x87c000 thru 0x87ffff (0x4000 bytes) 000479[0] 1712275781.501485 mapwrite 0x219000 thru 0x22089d (0x789e bytes) 000483[0] 1712275781.552075 write_OD 0x48c000 thru 0x494fff (0x9000 bytes) 000485[0] 1712275781.577354 trunc from 0x880000 to 0x8b3adc (0x33add bytes) 000486[1] 1712275781.584746 trunc from 0x8b3add to 0x4fca50 (0x3b708c bytes) 000488[0] 1712275781.596974 trunc from 0x4fca51 to 0x864a23 (0x367fd3 bytes) 000491[0] 1712275781.612322 trunc from 0x864a24 to 0x21e601 (0x646422 bytes) 000494[1] 1712275781.617321 mapread 0x1be000 thru 0x1c53cd (0x73ce bytes) 000495[1] 1712275781.633034 trunc from 0x21e602 to 0x2b3816 (0x95215 bytes) 000497[1] 1712275781.648345 read_OD 0x074000 thru 0x074fff (0x1000 bytes) 000498[0] 1712275781.653655 write_OD 0x404000 thru 0x408fff (0x5000 bytes) 000500[1] 1712275781.674446 mapwrite 0x2a7000 thru 0x2aef92 (0x7f93 bytes) All operations completed A-OK! PASS 16a (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 20:09:44 (1712275784) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.144319 s, 58.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.126027 s, 66.6 MB/s Chance of close/open is 1 in 50 Seed set to 5785 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1712275785.375609 mapwrite 0x3177ce thru 0x319522 (0x1d55 bytes) 000003[1] 1712275785.416648 write 0x6b7fb5 thru 0x6b89cb (0xa17 bytes) truncating to largest ever: 0x356601 000004[0] 1712275785.422631 trunc from 0x6b89cc to 0x356600 (0x3623cb bytes) 000009[1] 1712275785.441085 mapwrite 0x6299b9 thru 0x62b676 (0x1cbe bytes) 000011[0] 1712275785.480105 trunc from 0x62b677 to 0x34e8ff (0x2dcd77 bytes) 000014[1] 1712275785.492664 trunc from 0x34e900 to 0x0431b0 (0x30b74f bytes) 000016[1] 1712275785.505868 mapread 0x00c7bf thru 0x00e39a (0x1bdc bytes) 000017[0] 1712275785.511028 write 0x8dae9c thru 0x8dbcf1 (0xe56 bytes) 000019[0] 1712275785.522343 read 0x2e3c59 thru 0x2e435a (0x702 bytes) 000020[1] 1712275785.527064 write 0x357ae8 thru 0x358028 (0x541 bytes) 000022[1] 1712275785.539551 mapread 0x0ac49c thru 0x0ace69 (0x9ce bytes) 000025[1] 1712275785.545236 mapread 0x74c7ba thru 0x74e711 (0x1f58 bytes) 000026[0] 1712275785.553171 read 0x0825c8 thru 0x08451d (0x1f56 bytes) 000027[0] 1712275785.579271 mapwrite 0x802880 thru 0x803768 (0xee9 bytes) 000028[1] 1712275785.605894 write 0x982c5a thru 0x9842a0 (0x1647 bytes) 000029[0] 1712275785.608401 mapwrite 0x194875 thru 0x1959ac (0x1138 bytes) 000031[0] 1712275785.636193 trunc from 0x9842a1 to 0x191703 (0x7f2b9d bytes) 000032[0] 1712275785.646979 write 0x8995c3 thru 0x89a31f (0xd5d bytes) 000034[1] 1712275785.653805 mapread 0x18810a thru 0x1888c4 (0x7bb bytes) 000035[1] 1712275785.661744 mapwrite 0x853293 thru 0x854d89 (0x1af7 bytes) 000037[1] 1712275785.688354 mapread 0x1c0d1c thru 0x1c12dd (0x5c2 bytes) 000041[0] 1712275785.704950 write 0x1c1d9d thru 0x1c222e (0x492 bytes) 000043[1] 1712275785.726080 read 0x528717 thru 0x5292e5 (0xbcf bytes) 000044[0] 1712275785.739433 write 0x8dc043 thru 0x8dd468 (0x1426 bytes) 000047[0] 1712275785.744039 read 0x0ccfb2 thru 0x0ce577 (0x15c6 bytes) 000048[0] 1712275785.772360 write 0x53d0e1 thru 0x53d5d1 (0x4f1 bytes) 000049[0] 1712275785.788687 read 0x0a872d thru 0x0aa422 (0x1cf6 bytes) 000050[0] 1712275785.792161 write 0x07a461 thru 0x07b9fd (0x159d bytes) truncating to largest ever: 0x5dcc60 000051[0] 1712275785.792654 trunc from 0x8dd469 to 0x5dcc5f (0x300809 bytes) 000052[1] 1712275785.801555 write 0x4851fb thru 0x48642b (0x1231 bytes) 000054[0] 1712275785.813993 trunc from 0x5dcc60 to 0x01ec21 (0x5be03e bytes) 000056[0] 1712275785.828673 trunc from 0x01ec22 to 0x0070f6 (0x17b2b bytes) 000059[1] 1712275785.838073 mapwrite 0x691be2 thru 0x693670 (0x1a8f bytes) 000060[1] 1712275785.866981 mapwrite 0x7fcbe8 thru 0x7fd858 (0xc71 bytes) 000061[1] 1712275785.889571 read 0x19ea5b thru 0x19f452 (0x9f8 bytes) 000063[1] 1712275785.916148 mapread 0x7dc2b0 thru 0x7dc36a (0xbb bytes) 000065[0] 1712275785.937027 mapread 0x6af304 thru 0x6b00c4 (0xdc1 bytes) 000070[1] 1712275785.952778 write 0x1f341f thru 0x1f52c5 (0x1ea7 bytes) 000071[0] 1712275785.953235 write 0x5fdeef thru 0x5ff04e (0x1160 bytes) 000073[0] 1712275785.965206 mapread 0x1d5d84 thru 0x1d7633 (0x18b0 bytes) 000076[0] 1712275786.002212 trunc from 0x7fd859 to 0x5dcb7b (0x220cdd bytes) 000077[1] 1712275786.013723 mapread 0x1116f6 thru 0x111af9 (0x404 bytes) 000078[0] 1712275786.023615 mapwrite 0x4c49ac thru 0x4c5d65 (0x13ba bytes) 000080[1] 1712275786.053094 trunc from 0x5dcb7c to 0x0c71ea (0x515991 bytes) 000083[1] 1712275786.067621 trunc from 0x0c71eb to 0x14cfe6 (0x85dfc bytes) 000084[1] 1712275786.072145 read 0x0bf891 thru 0x0c02eb (0xa5b bytes) 000091[0] 1712275786.089372 read 0x1346f3 thru 0x1360a3 (0x19b1 bytes) 000095[1] 1712275786.105970 mapread 0x073312 thru 0x073a5d (0x74c bytes) 000096[0] 1712275786.107591 mapread 0x06fb94 thru 0x0711d6 (0x1643 bytes) 000097[1] 1712275786.112397 trunc from 0x14cfe7 to 0x59382d (0x446847 bytes) 000099[1] 1712275786.126759 read 0x42b327 thru 0x42c562 (0x123c bytes) 000100[0] 1712275786.151983 mapread 0x559c0d thru 0x559d54 (0x148 bytes) 000102[0] 1712275786.160681 read 0x539f15 thru 0x53b666 (0x1752 bytes) 000103[0] 1712275786.167149 mapwrite 0x8b29bb thru 0x8b3e48 (0x148e bytes) 000105[0] 1712275786.229083 mapwrite 0x612e10 thru 0x613206 (0x3f7 bytes) 000108[0] 1712275786.252231 trunc from 0x8b3e49 to 0x5b0d73 (0x3030d5 bytes) 000111[1] 1712275786.269461 write 0x0c497f thru 0x0c5a45 (0x10c7 bytes) 000113[0] 1712275786.285359 trunc from 0x5b0d74 to 0x1dd31b (0x3d3a58 bytes) 000114[0] 1712275786.298074 mapread 0x08027e thru 0x081588 (0x130b bytes) 000119[1] 1712275786.314093 mapwrite 0x9f9231 thru 0x9f9e62 (0xc32 bytes) 000123[0] 1712275786.354941 write 0x9d5b1c thru 0x9d7349 (0x182e bytes) 000125[0] 1712275786.375927 mapwrite 0x0763da thru 0x077c9a (0x18c1 bytes) 000126[1] 1712275786.398186 mapread 0x7daee3 thru 0x7dc382 (0x14a0 bytes) 000127[0] 1712275786.403255 write 0x09fd4f thru 0x0a0d31 (0xfe3 bytes) truncating to largest ever: 0x98f3e4 000128[1] 1712275786.409259 trunc from 0x9f9e63 to 0x98f3e3 (0x6aa7f bytes) 000129[1] 1712275786.422354 read 0x549692 thru 0x54ae77 (0x17e6 bytes) 000130[0] 1712275786.449636 read 0x35f386 thru 0x35fec2 (0xb3d bytes) 000132[0] 1712275786.487574 trunc from 0x98f3e4 to 0x2a3d78 (0x6eb66b bytes) 000135[1] 1712275786.513635 mapread 0x0c9156 thru 0x0ca7ac (0x1657 bytes) 000139[0] 1712275786.525032 trunc from 0x2a3d79 to 0x478967 (0x1d4bef bytes) 000141[1] 1712275786.538908 write 0x974baf thru 0x974fcf (0x421 bytes) 000145[0] 1712275786.558164 read 0x570dbf thru 0x571cfb (0xf3d bytes) 000150[0] 1712275786.581559 mapwrite 0x9a45f5 thru 0x9a64be (0x1eca bytes) 000151[0] 1712275786.657725 read 0x7b6e2f thru 0x7b7f96 (0x1168 bytes) 000155[0] 1712275786.666950 mapread 0x4fb6e6 thru 0x4fd26b (0x1b86 bytes) 000156[1] 1712275786.671528 write 0x893326 thru 0x894d3f (0x1a1a bytes) 000158[1] 1712275786.690522 mapwrite 0x05ef4e thru 0x060311 (0x13c4 bytes) 000161[0] 1712275786.723429 trunc from 0x9a64bf to 0x3b4426 (0x5f2098 bytes) 000163[1] 1712275786.735003 mapwrite 0x125284 thru 0x1271c9 (0x1f46 bytes) 000164[0] 1712275786.762889 mapread 0x38e9fe thru 0x390514 (0x1b17 bytes) 000165[0] 1712275786.787055 mapwrite 0x5f9cc0 thru 0x5f9ef6 (0x237 bytes) 000166[1] 1712275786.815503 mapread 0x41f401 thru 0x41fef2 (0xaf2 bytes) 000170[0] 1712275786.830598 trunc from 0x5f9ef7 to 0x202edd (0x3f7019 bytes) 000172[1] 1712275786.845258 mapread 0x11c30f thru 0x11cd59 (0xa4b bytes) 000173[1] 1712275786.854386 trunc from 0x202ede to 0x8d4cc6 (0x6d1de9 bytes) 000176[1] 1712275786.869903 mapwrite 0x969217 thru 0x96ada5 (0x1b8f bytes) 000177[1] 1712275786.896181 trunc from 0x96ada6 to 0x52d97f (0x43d426 bytes) 000178[0] 1712275786.901120 mapread 0x37d699 thru 0x37f31c (0x1c84 bytes) 000182[1] 1712275786.920529 write 0x0d7443 thru 0x0d82de (0xe9c bytes) 000183[1] 1712275786.924712 mapwrite 0x86f963 thru 0x8717c8 (0x1e66 bytes) 000184[1] 1712275786.953456 write 0x80e067 thru 0x80f81e (0x17b8 bytes) 000186[0] 1712275786.956574 mapread 0x773330 thru 0x773387 (0x58 bytes) 000187[1] 1712275786.986687 mapread 0x77c97f thru 0x77dda3 (0x1425 bytes) 000188[1] 1712275786.989609 write 0x834cf3 thru 0x83694d (0x1c5b bytes) 000189[0] 1712275786.997447 trunc from 0x8717c9 to 0x93c1a0 (0xca9d8 bytes) 000190[0] 1712275787.013181 mapread 0x80343e thru 0x804065 (0xc28 bytes) 000191[1] 1712275787.029470 mapwrite 0x64633f thru 0x647a26 (0x16e8 bytes) 000194[0] 1712275787.062083 write 0x0d11db thru 0x0d16c9 (0x4ef bytes) 000195[0] 1712275787.067362 mapread 0x3bd935 thru 0x3be6ac (0xd78 bytes) 000196[0] 1712275787.075193 mapwrite 0x687286 thru 0x688325 (0x10a0 bytes) 000197[1] 1712275787.097911 mapwrite 0x9684e4 thru 0x9699bb (0x14d8 bytes) 000199[1] 1712275787.131890 mapread 0x7486ab thru 0x7497a7 (0x10fd bytes) 000200[0] 1712275787.139077 mapread 0x7578b7 thru 0x75925c (0x19a6 bytes) 000202[1] 1712275787.153100 mapread 0x267b65 thru 0x2687be (0xc5a bytes) 000204[0] 1712275787.162557 write 0x6faad1 thru 0x6fab30 (0x60 bytes) 000207[0] 1712275787.178049 write 0x2ac4e7 thru 0x2adf70 (0x1a8a bytes) 000209[1] 1712275787.189184 trunc from 0x9699bc to 0x367761 (0x60225a bytes) 000210[1] 1712275787.201299 read 0x0f4e37 thru 0x0f5253 (0x41d bytes) 000211[0] 1712275787.216042 write 0x93e91e thru 0x93fa52 (0x1135 bytes) 000212[1] 1712275787.228218 mapread 0x0bc74d thru 0x0bcaf4 (0x3a8 bytes) 000213[0] 1712275787.243344 read 0x731704 thru 0x7331a6 (0x1aa3 bytes) 000214[0] 1712275787.272267 trunc from 0x93fa53 to 0x366b59 (0x5d8ef9 bytes) 000217[1] 1712275787.296999 write 0x05b466 thru 0x05c9f8 (0x1593 bytes) 000219[0] 1712275787.313049 trunc from 0x366b5a to 0x625fed (0x2bf494 bytes) 000220[0] 1712275787.319942 trunc from 0x625fee to 0x05ff8d (0x5c6060 bytes) 000221[0] 1712275787.323013 read 0x0289c8 thru 0x02a729 (0x1d62 bytes) 000222[0] 1712275787.331688 read 0x02237a thru 0x022b0c (0x793 bytes) 000225[0] 1712275787.336028 mapread 0x04bb36 thru 0x04d849 (0x1d14 bytes) 000226[1] 1712275787.340016 trunc from 0x05ff8e to 0x3c5960 (0x3659d3 bytes) 000227[1] 1712275787.348736 mapread 0x29ad00 thru 0x29bde6 (0x10e7 bytes) 000229[1] 1712275787.355960 read 0x25eb16 thru 0x25f968 (0xe53 bytes) 000230[0] 1712275787.357718 mapread 0x387fb2 thru 0x389da4 (0x1df3 bytes) 000231[1] 1712275787.385123 mapwrite 0x575f2e thru 0x577853 (0x1926 bytes) 000233[0] 1712275787.418221 mapwrite 0x8cdec3 thru 0x8cf3fa (0x1538 bytes) 000235[1] 1712275787.467869 write 0x5febce thru 0x5fef86 (0x3b9 bytes) 000239[0] 1712275787.488592 mapread 0x8a8e59 thru 0x8a93b9 (0x561 bytes) 000240[1] 1712275787.493818 write 0x98aa0c thru 0x98c674 (0x1c69 bytes) 000241[0] 1712275787.497667 write 0x28efd1 thru 0x290ee1 (0x1f11 bytes) 000243[1] 1712275787.515613 write 0x5ce9f0 thru 0x5ceca1 (0x2b2 bytes) 000245[1] 1712275787.524762 mapwrite 0x208fa3 thru 0x20a970 (0x19ce bytes) 000249[1] 1712275787.553473 write 0x1d512a thru 0x1d594c (0x823 bytes) 000251[1] 1712275787.560073 read 0x0bdc26 thru 0x0bf177 (0x1552 bytes) 000255[0] 1712275787.597663 trunc from 0x98c675 to 0x26e600 (0x71e074 bytes) 000258[0] 1712275787.621869 mapwrite 0x1a46ed thru 0x1a4e65 (0x779 bytes) 000260[0] 1712275787.645172 trunc from 0x26e601 to 0x139af2 (0x134b0e bytes) 000261[0] 1712275787.653275 read 0x063bd6 thru 0x06596a (0x1d95 bytes) 000262[0] 1712275787.664979 read 0x03ccdf thru 0x03cfb8 (0x2da bytes) 000263[1] 1712275787.668227 mapread 0x137509 thru 0x137e0b (0x903 bytes) 000264[1] 1712275787.676699 read 0x1298ea thru 0x129b78 (0x28f bytes) 000265[1] 1712275787.685493 write 0x2b4bc9 thru 0x2b4d19 (0x151 bytes) 000267[0] 1712275787.695522 mapwrite 0x4d83ad thru 0x4d8807 (0x45b bytes) 000268[0] 1712275787.733591 mapwrite 0x81052a thru 0x8119a8 (0x147f bytes) 000269[1] 1712275787.764117 write 0x09b025 thru 0x09be3b (0xe17 bytes) 000270[0] 1712275787.790451 mapwrite 0x5fe1f4 thru 0x600036 (0x1e43 bytes) 000271[0] 1712275787.808046 write 0x3248ca thru 0x324bc0 (0x2f7 bytes) 000273[1] 1712275787.818474 trunc from 0x8119a9 to 0x26dc72 (0x5a3d36 bytes) truncating to largest ever: 0x9bd507 000281[1] 1712275787.845706 trunc from 0x26dc73 to 0x9bd506 (0x74f894 bytes) 000282[0] 1712275787.855732 mapwrite 0x2564b7 thru 0x257274 (0xdbe bytes) 000283[1] 1712275787.892917 read 0x2694a7 thru 0x26a9da (0x1534 bytes) 000284[1] 1712275787.926828 read 0x4cc344 thru 0x4cd9c2 (0x167f bytes) 000285[1] 1712275787.940971 write 0x41cbf0 thru 0x41d879 (0xc8a bytes) 000286[1] 1712275787.944951 mapread 0x7c00bd thru 0x7c05f8 (0x53c bytes) 000288[1] 1712275787.945622 write 0x8248a4 thru 0x825000 (0x75d bytes) 000289[0] 1712275787.974673 trunc from 0x9bd507 to 0x137bf3 (0x885913 bytes) 000292[1] 1712275788.003352 read 0x0e940c thru 0x0ea808 (0x13fd bytes) 000293[1] 1712275788.017528 mapread 0x0ade17 thru 0x0ae268 (0x452 bytes) 000294[1] 1712275788.020374 read 0x082465 thru 0x083c48 (0x17e4 bytes) 000295[0] 1712275788.020707 write 0x5a29e6 thru 0x5a3ecd (0x14e8 bytes) 000296[0] 1712275788.025940 trunc from 0x5a3ece to 0x82f433 (0x28b566 bytes) 000297[1] 1712275788.035458 write 0x568391 thru 0x569694 (0x1304 bytes) 000298[0] 1712275788.045734 write 0x2cb16f thru 0x2cbc74 (0xb06 bytes) 000300[1] 1712275788.053955 read 0x781984 thru 0x782cee (0x136b bytes) 000301[0] 1712275788.059803 mapread 0x2d6d07 thru 0x2d8bc3 (0x1ebd bytes) 000304[1] 1712275788.074499 mapwrite 0x35c733 thru 0x35e40c (0x1cda bytes) 000305[1] 1712275788.114347 mapread 0x0f33f6 thru 0x0f3cc4 (0x8cf bytes) truncating to largest ever: 0x9c53b0 000308[0] 1712275788.118835 trunc from 0x82f434 to 0x9c53af (0x195f7c bytes) 000309[1] 1712275788.133932 write 0x1367e1 thru 0x1386c2 (0x1ee2 bytes) 000310[0] 1712275788.143080 write 0x6a8264 thru 0x6a89ff (0x79c bytes) 000314[1] 1712275788.158104 mapread 0x473666 thru 0x47435f (0xcfa bytes) 000315[0] 1712275788.188359 read 0x06b7fa thru 0x06ca2e (0x1235 bytes) 000317[1] 1712275788.213606 trunc from 0x9c53b0 to 0x1bf03b (0x806374 bytes) 000318[1] 1712275788.233900 write 0x4eb9c1 thru 0x4ed3af (0x19ef bytes) 000321[0] 1712275788.236479 read 0x189cd9 thru 0x18a80e (0xb36 bytes) 000325[1] 1712275788.254207 read 0x4d9ac1 thru 0x4da68c (0xbcc bytes) 000327[1] 1712275788.268271 mapwrite 0x9ebbdc thru 0x9ed759 (0x1b7e bytes) 000328[0] 1712275788.326830 write 0x7c23c8 thru 0x7c27cf (0x408 bytes) 000330[0] 1712275788.343876 mapwrite 0x829542 thru 0x829716 (0x1d5 bytes) 000331[0] 1712275788.379295 trunc from 0x9ed75a to 0x18bc7f (0x861ada bytes) 000334[1] 1712275788.386750 mapwrite 0x99b0c2 thru 0x99bc0a (0xb49 bytes) 000335[1] 1712275788.412864 mapread 0x868acc thru 0x869acd (0x1002 bytes) 000338[1] 1712275788.420990 read 0x5ac265 thru 0x5ad01a (0xdb6 bytes) 000341[0] 1712275788.463432 write 0x4f3198 thru 0x4f346c (0x2d5 bytes) 000342[0] 1712275788.495215 mapwrite 0x02263a thru 0x022d31 (0x6f8 bytes) 000344[0] 1712275788.533042 mapread 0x3c9aa2 thru 0x3ca820 (0xd7f bytes) 000345[1] 1712275788.536019 read 0x54ed3f thru 0x550a47 (0x1d09 bytes) 000347[0] 1712275788.553464 mapwrite 0x0d3564 thru 0x0d4dad (0x184a bytes) 000349[0] 1712275788.574783 write 0x931b03 thru 0x932a84 (0xf82 bytes) 000350[0] 1712275788.578243 mapread 0x6f0739 thru 0x6f0f00 (0x7c8 bytes) 000351[1] 1712275788.582873 mapread 0x1722ef thru 0x173e26 (0x1b38 bytes) 000352[0] 1712275788.611121 mapwrite 0x4099f8 thru 0x409bc4 (0x1cd bytes) 000354[0] 1712275788.627022 trunc from 0x99bc0b to 0x3ef451 (0x5ac7b9 bytes) 000355[1] 1712275788.638997 trunc from 0x3ef452 to 0x53eb8d (0x14f73c bytes) 000356[1] 1712275788.651769 read 0x2b9654 thru 0x2bacc7 (0x1674 bytes) 000358[1] 1712275788.675443 read 0x3aa737 thru 0x3aac58 (0x522 bytes) 000359[1] 1712275788.675839 mapwrite 0x60218e thru 0x602fe1 (0xe54 bytes) 000360[1] 1712275788.703030 trunc from 0x602fe2 to 0x98aae8 (0x387b07 bytes) 000361[0] 1712275788.712855 write 0x48c612 thru 0x48cc97 (0x686 bytes) 000362[1] 1712275788.722622 write 0x82151b thru 0x8224d0 (0xfb6 bytes) 000363[0] 1712275788.728808 mapread 0x43e168 thru 0x43fd03 (0x1b9c bytes) 000364[1] 1712275788.737249 write 0x5bd86a thru 0x5be33f (0xad6 bytes) 000367[1] 1712275788.773317 trunc from 0x98aae9 to 0x974d6a (0x15d7e bytes) 000369[0] 1712275788.784006 mapwrite 0x03ec01 thru 0x04044b (0x184b bytes) 000371[0] 1712275788.818039 mapread 0x79f669 thru 0x7a0784 (0x111c bytes) 000372[1] 1712275788.833530 mapwrite 0x351249 thru 0x35313b (0x1ef3 bytes) 000373[1] 1712275788.872579 trunc from 0x974d6b to 0x09fa19 (0x8d5351 bytes) 000378[0] 1712275788.889147 write 0x0d7304 thru 0x0d8a58 (0x1755 bytes) 000380[1] 1712275788.899894 mapread 0x054585 thru 0x0553d3 (0xe4f bytes) 000382[1] 1712275788.911294 trunc from 0x0d8a59 to 0x64fb91 (0x577139 bytes) 000383[0] 1712275788.919311 read 0x187f62 thru 0x189e6f (0x1f0e bytes) 000384[0] 1712275788.942678 mapwrite 0x053cd7 thru 0x054b6f (0xe99 bytes) 000385[1] 1712275788.981050 read 0x12d4ad thru 0x12d877 (0x3cb bytes) 000389[1] 1712275789.011029 read 0x292219 thru 0x292fef (0xdd7 bytes) truncating to largest ever: 0x9e0f15 000391[1] 1712275789.018299 trunc from 0x64fb92 to 0x9e0f14 (0x391383 bytes) 000394[1] 1712275789.034680 write 0x873a06 thru 0x873f5c (0x557 bytes) 000396[1] 1712275789.057294 mapread 0x9dcf66 thru 0x9ddc5f (0xcfa bytes) 000397[1] 1712275789.080175 read 0x8915d7 thru 0x891ac2 (0x4ec bytes) 000399[1] 1712275789.084040 mapread 0x075b3a thru 0x076e0e (0x12d5 bytes) 000401[0] 1712275789.084716 read 0x841cdb thru 0x843aa7 (0x1dcd bytes) 000402[1] 1712275789.113795 trunc from 0x9e0f15 to 0x5d700f (0x409f05 bytes) 000403[1] 1712275789.125914 read 0x2c0ef3 thru 0x2c14ab (0x5b9 bytes) 000404[1] 1712275789.149550 mapwrite 0x03662f thru 0x036756 (0x128 bytes) 000405[0] 1712275789.168094 trunc from 0x5d7010 to 0x0eac24 (0x4ec3eb bytes) 000406[1] 1712275789.184304 write 0x8ae845 thru 0x8aedfb (0x5b7 bytes) 000407[1] 1712275789.191524 mapwrite 0x924222 thru 0x924fd7 (0xdb6 bytes) 000408[0] 1712275789.215567 read 0x22346d thru 0x224cf4 (0x1888 bytes) 000409[1] 1712275789.238772 trunc from 0x924fd8 to 0x1395ac (0x7eba2b bytes) 000414[0] 1712275789.257817 mapread 0x080a14 thru 0x08147c (0xa69 bytes) 000420[0] 1712275789.265297 mapwrite 0x10ab16 thru 0x10afb0 (0x49b bytes) 000421[1] 1712275789.287168 mapread 0x0995b7 thru 0x09a323 (0xd6d bytes) 000422[1] 1712275789.292710 trunc from 0x1395ad to 0x24b091 (0x111ae5 bytes) 000425[0] 1712275789.305080 trunc from 0x24b092 to 0x3617eb (0x11675a bytes) 000428[0] 1712275789.321907 mapwrite 0x9472a0 thru 0x9488f3 (0x1654 bytes) 000432[1] 1712275789.375093 mapwrite 0x31a9e9 thru 0x31be26 (0x143e bytes) 000433[1] 1712275789.406982 write 0x6d37cf thru 0x6d474f (0xf81 bytes) 000435[1] 1712275789.415801 mapwrite 0x7ab5fb thru 0x7ab628 (0x2e bytes) 000437[1] 1712275789.437738 mapwrite 0x1a876c thru 0x1a8b88 (0x41d bytes) 000439[0] 1712275789.457914 write 0x4975dc thru 0x49762e (0x53 bytes) 000440[0] 1712275789.464822 mapwrite 0x421813 thru 0x4236f1 (0x1edf bytes) 000441[1] 1712275789.489714 trunc from 0x9488f4 to 0x01c25b (0x92c698 bytes) 000444[0] 1712275789.502642 read 0x019c88 thru 0x019f01 (0x27a bytes) 000445[1] 1712275789.508160 write 0x618d8b thru 0x61a63e (0x18b4 bytes) 000446[1] 1712275789.512954 mapwrite 0x2f2d0a thru 0x2f488b (0x1b82 bytes) 000448[0] 1712275789.536967 mapwrite 0x1260e4 thru 0x127cf6 (0x1c13 bytes) 000449[0] 1712275789.575567 trunc from 0x61a63f to 0x8e1df7 (0x2c77b9 bytes) 000450[0] 1712275789.586988 write 0x13ca4f thru 0x13e628 (0x1bda bytes) 000451[1] 1712275789.590607 write 0x56ca1f thru 0x56e6ca (0x1cac bytes) 000452[0] 1712275789.599179 mapread 0x466cf4 thru 0x46850f (0x181c bytes) 000453[1] 1712275789.635275 read 0x28bbd7 thru 0x28cedd (0x1307 bytes) 000455[0] 1712275789.672132 trunc from 0x8e1df8 to 0x74f55b (0x19289c bytes) 000457[0] 1712275789.695592 read 0x5014bf thru 0x503109 (0x1c4b bytes) 000460[0] 1712275789.729953 mapwrite 0x5a5c14 thru 0x5a70a0 (0x148d bytes) 000462[0] 1712275789.745119 mapwrite 0x60b203 thru 0x60c23b (0x1039 bytes) 000469[0] 1712275789.775134 mapread 0x6e4f23 thru 0x6e65c4 (0x16a2 bytes) 000470[0] 1712275789.775574 trunc from 0x74f55c to 0x5232e9 (0x22c272 bytes) 000472[1] 1712275789.781816 mapwrite 0x04b8f7 thru 0x04c13a (0x844 bytes) 000473[1] 1712275789.807305 mapread 0x3b441c thru 0x3b4677 (0x25c bytes) 000474[0] 1712275789.813220 write 0x363e2f thru 0x365c56 (0x1e28 bytes) 000478[1] 1712275789.825089 write 0x74fbb9 thru 0x74fd39 (0x181 bytes) 000479[0] 1712275789.832093 read 0x57c518 thru 0x57dc34 (0x171d bytes) 000480[1] 1712275789.865703 write 0x47ceef thru 0x47d951 (0xa63 bytes) 000483[0] 1712275789.882926 trunc from 0x74fd3a to 0x747d27 (0x8012 bytes) 000486[0] 1712275789.894789 write 0x6ef90b thru 0x6f18ea (0x1fe0 bytes) 000487[1] 1712275789.897436 trunc from 0x747d28 to 0x7a0aa5 (0x58d7e bytes) 000491[1] 1712275789.916840 mapwrite 0x8df565 thru 0x8dfb59 (0x5f5 bytes) 000495[1] 1712275789.956926 mapwrite 0x955df3 thru 0x957483 (0x1691 bytes) 000497[0] 1712275789.982491 trunc from 0x957484 to 0x375f62 (0x5e1521 bytes) 000498[0] 1712275789.996467 trunc from 0x375f63 to 0x68dfc5 (0x318063 bytes) 000500[1] 1712275790.005899 mapwrite 0x88876d thru 0x88a2cd (0x1b61 bytes) All operations completed A-OK! PASS 16b (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 20:09:52 (1712275792) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 20:09:55 (1712275795) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.126668 s, 66.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.123912 s, 67.7 MB/s ldlm.namespaces.MGC192.168.204.132@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a9c2d800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012a4ae000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800a9c2d800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012a4ae000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800a9c2d800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012a4ae000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.02378 s, 34.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.723586 s, 145 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.565035 s, 186 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.24382 s, 84.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.74614 s, 60.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.751738 s, 139 MB/s PASS 16d (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 20:10:07 (1712275807) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.058294 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270044 s, 38.8 MB/s PASS 16e (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 20:10:10 (1712275810) TIMEOUT PASS 16f (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 20:10:33 (1712275833) TIMEOUT PASS 16g (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16h: mmap read after truncate file ======= 20:10:56 (1712275856) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16i: read after truncate file ============ 20:10:59 (1712275859) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16j: race dio with buffered i/o ========== 20:11:02 (1712275862) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.32838 s, 79.8 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.241421 s, 109 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.506811 s, 51.7 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.144549 s, 181 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.496256 s, 52.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.161846 s, 162 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.563258 s, 46.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.197682 s, 133 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.251612 s, 104 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 11.5368 s, 2.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 11.5725 s, 2.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.18685 s, 140 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.255403 s, 103 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.593392 s, 44.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.627644 s, 41.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.237105 s, 111 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.308795 s, 84.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.680433 s, 38.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.735059 s, 35.7 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (19s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 20:11:22 (1712275882) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Thu Apr 4 20:11:40 EDT 2024 All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Thu Apr 4 20:11:41 EDT 2024 All operations completed A-OK! + fsx 3 done: rc=0 Thu Apr 4 20:11:41 EDT 2024 + fsx 4 done: rc=0 Thu Apr 4 20:11:41 EDT 2024 + fsx 5 done: rc=0 Thu Apr 4 20:11:41 EDT 2024 PASS 16k (23s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 17: resource creation/LVB creation race ========================================================================= 20:11:46 (1712275906) fail_loc=0x8000030a PASS 17 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 18: mmap sanity check =========================================================================================== 20:11:52 (1712275912) mmap test1: basic mmap operation (PASS, 0.019155s) mmap test2: MAP_PRIVATE not write back (PASS, 0.013625s) mmap test3: concurrent mmap ops on two nodes (PASS, 8.1482s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0702s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.026804s) mmap test6: check mmap write/read content on two nodes (PASS, 0.028481s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0113s) PASS 18 (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 19: test concurrent uncached read races ========================================================================= 20:12:07 (1712275927) oleg432-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg432-client: oleg432-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 20: test extra readahead page left in cache ============================================================== 20:12:10 (1712275930) PASS 20 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 20:12:13 (1712275933) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 23: others should see updated atime while another read============================================================== 20:12:16 (1712275936) now is 1712275937 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 new atime is 1712275998 PASS 23 (63s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 20:13:20 (1712276000) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.1K 401 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 402 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.2K 386 228.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 401 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 402 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.2K 386 228.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 11.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 32.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 43.0M 7.1G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117905 401 117504 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117234 402 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234722 386 234336 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.5K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 401 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 402 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.2K 386 228.8K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre[MDT:0] filesystem_summary: 502242 386 501856 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502242 386 501856 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117905 401 117504 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117234 402 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234722 386 234336 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 44032 7494656 1% /mnt/lustre2 PASS 24a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24b: lfs df should show both filesystems ========================================================================= 20:13:23 (1712276003) PASS 24b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 20:13:26 (1712276006) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 20:13:30 (1712276010) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26a: allow mtime to get older ============ 20:13:32 (1712276012) utime: good mknod times 1712276011 <= 1712276012 <= 1712276012 for /mnt/lustre/f26a utime: good mknod times 1712276011 <= 1712276012 <= 1712276012 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26b: sync mtime between ost and mds ====== 20:13:36 (1712276016) PASS 26b (4s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 30: recreate file race =================== 20:13:42 (1712276022) 0 PASS 30 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 20:13:47 (1712276027) fail_loc=0x314 PASS 31a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 20:13:51 (1712276031) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg432-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.132@tcp:/lustre 7542784 44032 7494656 1% /mnt/lustre2 PASS 31b (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31r: open-rename(replace) race =========== 20:14:03 (1712276043) -rw-r--r-- 1 root root 0 Apr 4 20:14 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: failed to access '/mnt/lustre2/f31r.sanityn.target': Input/output error PASS 31r (4s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_33a skipping SLOW test 33a debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 20:14:09 (1712276049) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 20:14:12 (1712276052) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33d: dependent transactions should trigger COS ========================================================== 20:14:14 (1712276054) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 20:14:16 (1712276056) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 34: no lock timeout under IO ============= 20:14:18 (1712276058) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg432-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c2d800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012a4ae000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9c2d800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012a4ae000.ost_server_uuid in IDLE state after 0 sec oleg432-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c2d800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012a4ae000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800a9c2d800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012a4ae000.ost_server_uuid in FULL state after 0 sec oleg432-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c2d800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012a4ae000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9c2d800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012a4ae000.ost_server_uuid in IDLE state after 0 sec oleg432-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c2d800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012a4ae000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800a9c2d800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012a4ae000.ost_server_uuid in FULL state after 0 sec oleg432-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c2d800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012a4ae000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800a9c2d800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012a4ae000.ost_server_uuid in IDLE state after 0 sec oleg432-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c2d800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012a4ae000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800a9c2d800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012a4ae000.ost_server_uuid in IDLE state after 0 sec PASS 34 (87s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 20:15:47 (1712276147) fail_loc=0x80000317 Race attempt 0 Wait for 24020 24159 for 60 sec... total: 4000 open/close in 8.96 seconds: 446.63 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.132@tcp:/lustre 7.2G 43M 7.2G 1% /mnt/lustre 192.168.204.132@tcp:/lustre 7.2G 43M 7.2G 1% /mnt/lustre2 PASS 35 (63s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 36: handle ESTALE/open-unlink correctly == 20:16:52 (1712276212) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg432-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.279732 s, 187 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7494656) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.259375 s, 202 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7492608) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.268889 s, 195 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.240199 s, 218 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7494656) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.26149 s, 201 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.267618 s, 196 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7494656) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.304939 s, 172 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7494656) after_dd(7437312) after(7488512) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.270253 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7488512) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.240631 s, 218 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7492608) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.269831 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7492608) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.292984 s, 179 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7492608) after_dd(7443456) after(7494656) PASS 36 (228s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 20:20:42 (1712276442) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 - create 8433 (time 1712276453.39 total 10.00 last 843.23) total: 10000 create in 11.79 seconds: 848.23 ops/second PASS 37 (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39a: file mtime does not change after rename ========================================================== 20:20:58 (1712276458) repeat after cancel_lru_locks PASS 39a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 20:21:02 (1712276462) repeat after cancel_lru_locks PASS 39b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39c: check truncate mtime update ================================================================================ 20:21:06 (1712276466) repeat after cancel_lru_locks PASS 39c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39d: sync write should update mtime ====== 20:21:10 (1712276470) fail_loc=0x411 fail_loc=0 PASS 39d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40a: pdirops: create vs others ======================================================================== 20:21:14 (1712276474) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40b: pdirops: open|create and others ======================================================================== 20:21:24 (1712276484) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40c: pdirops: link and others ======================================================================== 20:21:34 (1712276494) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40d: pdirops: unlink and others ======================================================================== 20:21:44 (1712276504) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40e: pdirops: rename and others ======================================================================== 20:21:52 (1712276512) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 20:22:00 (1712276520) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41b: pdirops: create vs create ======================================================================== 20:22:07 (1712276527) open: File exists Conflict PASS 41b (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41c: pdirops: create vs link ======================================================================== 20:22:13 (1712276533) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41d: pdirops: create vs unlink ======================================================================== 20:22:18 (1712276538) Conflict PASS 41d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 20:22:24 (1712276544) Conflict PASS 41e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 20:22:30 (1712276550) Conflict PASS 41f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41g: pdirops: create vs getattr ======================================================================== 20:22:36 (1712276556) Conflict PASS 41g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41h: pdirops: create vs readdir ======================================================================== 20:22:43 (1712276563) Conflict PASS 41h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41i: reint_open: create vs create ======== 20:22:49 (1712276569) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (698s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 20:34:29 (1712277269) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 20:34:36 (1712277276) open: File exists Conflict PASS 42b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 20:34:43 (1712277283) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 20:34:51 (1712277291) Conflict PASS 42d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 20:34:59 (1712277299) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 20:35:06 (1712277306) Conflict PASS 42f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 20:35:13 (1712277313) Conflict PASS 42g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 20:35:21 (1712277321) Conflict PASS 42h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 20:35:28 (1712277328) PASS 43a (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43b: pdirops: unlink vs create ======================================================================== 20:35:42 (1712277342) Conflict PASS 43b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43c: pdirops: unlink vs link ======================================================================== 20:35:49 (1712277349) Conflict PASS 43c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 20:35:56 (1712277356) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 20:36:04 (1712277364) Conflict PASS 43e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 20:36:11 (1712277371) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 20:36:18 (1712277378) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 20:36:26 (1712277386) Conflict PASS 43h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 20:36:33 (1712277393) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 20:36:36 (1712277396) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (42s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43k: unlink vs create ==================== 20:37:19 (1712277439) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (429s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 20:44:31 (1712277871) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu Apr 4 20:44:34 EDT 2024 PASS 44a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 20:44:37 (1712277877) open: File exists Conflict PASS 44b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 20:44:43 (1712277883) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 20:44:49 (1712277889) Conflict PASS 44d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 20:44:57 (1712277897) Conflict PASS 44e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 20:45:04 (1712277904) Conflict PASS 44f (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 20:45:12 (1712277912) Conflict PASS 44g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 20:45:19 (1712277919) Conflict PASS 44h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 20:45:27 (1712277927) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 20:45:30 (1712277930) PASS 45a (35s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45b: pdirops: rename src vs create ======================================================================== 20:46:07 (1712277967) Conflict PASS 45b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45c: pdirops: rename src vs link ======================================================================== 20:46:14 (1712277974) Conflict PASS 45c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 20:46:21 (1712277981) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 20:46:28 (1712277988) Conflict PASS 45e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 20:46:35 (1712277995) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 20:46:43 (1712278003) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 20:46:51 (1712278011) Conflict PASS 45h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 20:46:57 (1712278017) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45j: read vs rename ====================== 20:47:00 (1712278020) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (478s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 20:55:00 (1712278500) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46b: pdirops: link vs create ======================================================================== 20:55:07 (1712278507) open: File exists Conflict PASS 46b (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46c: pdirops: link vs link ======================================================================== 20:55:15 (1712278515) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46d: pdirops: link vs unlink ======================================================================== 20:55:22 (1712278522) Conflict PASS 46d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 20:55:30 (1712278530) Conflict PASS 46e (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 20:55:38 (1712278538) Conflict PASS 46f (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46g: pdirops: link vs getattr ======================================================================== 20:55:46 (1712278546) Conflict PASS 46g (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46h: pdirops: link vs readdir ======================================================================== 20:55:54 (1712278554) Conflict PASS 46h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46i: pdirops: link vs remote mkdir ======= 20:56:01 (1712278561) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 20:56:04 (1712278564) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47b: pdirops: remote mkdir vs create ===== 20:56:08 (1712278568) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47c: pdirops: remote mkdir vs link ======= 20:56:11 (1712278571) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 20:56:14 (1712278574) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 20:56:18 (1712278578) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 20:56:22 (1712278582) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 20:56:25 (1712278585) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 20:56:29 (1712278589) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.0330158 s, 310 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51a: layout lock: refresh layout should work ========================================================== 20:56:38 (1712278598) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00912911 s, 17.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3445: kill: (11458) - No such process PASS 51a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 20:56:45 (1712278605) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0035199 s, 291 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226832 s, 46.2 MB/s 3145728 PASS 51b (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 20:57:02 (1712278622) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00719166 s, 142 kB/s PASS 51c (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 20:57:11 (1712278631) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149685 s, 70.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 20:57:16 (1712278636) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2451 0x993 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2451 0x993 0x240000400 read lease(1) has applied. PASS 51e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 54: rename locking ======================= 20:57:23 (1712278643) llite.lustre-ffff8800a9c2d800.xattr_cache=1 llite.lustre-ffff88012a4ae000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505002 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 20:57:24.000000000 -0400 Modify: 2024-04-04 20:57:24.000000000 -0400 Change: 2024-04-04 20:57:24.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505005 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 20:57:29.000000000 -0400 Modify: 2024-04-04 20:57:30.000000000 -0400 Change: 2024-04-04 20:57:30.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505008 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 20:57:35.000000000 -0400 Modify: 2024-04-04 20:57:35.000000000 -0400 Change: 2024-04-04 20:57:35.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505007 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 20:57:35.000000000 -0400 Modify: 2024-04-04 20:57:35.000000000 -0400 Change: 2024-04-04 20:57:35.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 20:57:40.000000000 -0400 Modify: 2024-04-04 20:57:41.000000000 -0400 Change: 2024-04-04 20:57:41.000000000 -0400 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505010 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 20:57:40.000000000 -0400 Modify: 2024-04-04 20:57:41.000000000 -0400 Change: 2024-04-04 20:57:41.000000000 -0400 Birth: - PASS 54 (25s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55a: rename vs unlink target dir ========= 20:57:50 (1712278670) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55b: rename vs unlink source dir ========= 20:58:00 (1712278680) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55c: rename vs unlink orphan target dir == 20:58:10 (1712278690) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55d: rename file vs link ================= 20:58:25 (1712278705) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56a: test llverdev with single large stripe ========================================================== 20:58:37 (1712278717) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1712278717 chunksize: 4096 size: 16777216000000 write offset: 3635412992kB 0.903125 MB/s read offset: 3635412992kB inf MB/s llverdev: data verified successfully PASS 56a (17s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 20:58:56 (1712278736) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1712278738 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 3.59042 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (39s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 60: Verify data_version behaviour ======== 20:59:38 (1712278778) PASS 60 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70a: cd directory && rm directory ======== 20:59:42 (1712278782) PASS 70a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70b: remove files after calling rm_entry ========================================================== 20:59:47 (1712278787) PASS 70b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71a: correct file map just after write operation is finished ========================================================== 20:59:51 (1712278791) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00477074 s, 13.7 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 20:59:55 (1712278795) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00433811 s, 15.1 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 20:59:58 (1712278798) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 21:00:01 (1712278801) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00688452 s, 9.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 21:00:05 (1712278805) llite.lustre-ffff8800a9c2d800.xattr_cache=1 llite.lustre-ffff88012a4ae000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 21:00:09 (1712278809) llite.lustre-ffff8800a9c2d800.xattr_cache=1 llite.lustre-ffff88012a4ae000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 74: flock deadlock: different mounts ======================================================================== 21:00:14 (1712278814) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0043598 s, 235 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00463629 s, 221 kB/s 26104: get lock2 26105: get lock1 26105: done 26105: get lock2 26104: done 26104: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 26104: cannot get lock1: Resource deadlock avoided 26105: done 26105: exit rc=0 26104: done 26104: exit rc=0 PASS 74 (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 75: osc: upcall after unuse lock============================================================================= 21:00:22 (1712278822) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0226038 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138354 s, 75.8 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4031: 26852 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 76: Verify MDT open_files listing ======== 21:00:32 (1712278832) opening files: ........ closing files: ........ PASS 76 (24s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77a: check FIFO NRS policy =============== 21:00:58 (1712278858) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.122668 s, 137 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177903 s, 58.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0201442 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101469 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121858 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0672927 s, 15.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0683533 s, 15.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0730593 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0756243 s, 13.9 MB/s , 0.071902 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.081415 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0781349 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0834126 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0833277 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824772 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0868654 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0930488 s, 11.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a4ae000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff88012a4ae000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847268 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087841 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874567 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911604 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145724 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101142 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113024 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116421 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012188 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106775 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112413 s, 93.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0158644 s, 66.1 MB/s , 0.0143318 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130816 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120023 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164166 s, 63.9 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77b: check CRR-N NRS policy ============== 21:01:03 (1712278863) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0919638 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755729 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877833 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482481 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527581 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536939 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0547945 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0549966 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0550965 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0584275 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0556328 s, 18.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0612632 s, 17.1 MB/s , 0.0574746 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0641943 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0595756 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645658 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0666437 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111896 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161786 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0114978 s, 91.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0106004 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134053 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137301 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132414 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185071 s, 56.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0165859 s, 63.2 MB/s , 0.0141915 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143652 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146876 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172562 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173627 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017246 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165413 s, 63.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.141204 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146925 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186647 s, 56.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047147 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438893 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541653 s, 19.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0425312 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049621 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465083 s, 22.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0562037 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0546135 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412508 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439066 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498352 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0581099 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536859 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.058338 s, 18.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012a4ae000.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00838433 s, 125 MB/s , 0.00556257 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00702835 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00728394 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812536 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00363336 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090043 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00687937 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0056714 s, 185 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865371 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00857431 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894551 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886514 s, 118 MB/s 1048576 bytes (1.0 MB) copied, 0.0100797 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841774 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997493 s, 105 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77c: check ORR NRS policy ================ 21:01:08 (1712278868) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.151406 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115891 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216103 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254564 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127945 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330934 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360811 s, 29.1 MB/s 1+0 records in 1+0 records out , 0.0310954 s, 33.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0280528 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285751 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0400418 s, 26.2 MB/s , 0.0426409 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268323 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392489 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202658 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433885 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392474 s, 26.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0116326 s, 90.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00985055 s, 106 MB/s , 0.00784411 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924264 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109448 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125865 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108227 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106076 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127292 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123434 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108092 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119743 s, 87.6 MB/s 1+0 records in 1+0 records out , 0.0118573 s, 88.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0122634 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128249 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135934 s, 77.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0949349 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805425 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011378 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167483 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02793 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312272 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377823 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390987 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419905 s, 25.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332005 s, 31.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0483208 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476593 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441614 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406744 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388073 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422248 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468616 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00681581 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00687969 s, 152 MB/s , 0.00681756 s, 154 MB/s 1048576 bytes (1.0 MB) copied, 0.00756224 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00820874 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013373 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104175 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968978 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103366 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105925 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106955 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116461 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116206 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110228 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111248 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135365 s, 77.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.139123 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103256 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185943 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241031 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269745 s, 38.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0303863 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015941 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159603 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146149 s, 71.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0320115 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222978 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233335 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279433 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494254 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501004 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186703 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473582 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00692064 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00901816 s, 116 MB/s 1048576 bytes (1.0 MB) copied, 0.0100881 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658482 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00884209 s, 119 MB/s 1048576 bytes (1.0 MB) copied, 0.00966251 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888869 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901896 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985113 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110091 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116028 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116952 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112303 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107946 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119341 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143097 s, 73.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77d: check TRR nrs policy ================ 21:01:15 (1712278875) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.126686 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149568 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180987 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147292 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318167 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328608 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343377 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233067 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395379 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436498 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352546 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383356 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477297 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416127 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03981 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508155 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0558701 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812377 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123785 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101101 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00373218 s, 281 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997636 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146459 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103543 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121878 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118679 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117244 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012267 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120008 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126645 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126455 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013003 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132654 s, 79.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.15811 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0151004 s, 69.4 MB/s 1048576 bytes (1.0 MB) copied, 0.018907 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02522 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052009 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0661089 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0696421 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602025 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0537505 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0682446 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0784272 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0718525 s, 14.6 MB/s , 0.0782547 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0845611 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0700161 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828332 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0915141 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00937265 s, 112 MB/s , 0.00259554 s, 404 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00184054 s, 570 MB/s , 0.00878813 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125315 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106954 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107811 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126129 s, 83.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113173 s, 92.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0121904 s, 86.0 MB/s , 0.0140078 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125285 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136954 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150373 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136827 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155967 s, 67.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.140089 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159778 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222291 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0543475 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0629584 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0755197 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0826921 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0819062 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.07322 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0740858 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0948757 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0931774 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0939359 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0882615 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0930502 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0867561 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0839854 s, 12.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a4ae000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff88012a4ae000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00949142 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00713387 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966903 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00794386 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106913 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111674 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129752 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107401 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838723 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085698 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156308 s, 671 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0017641 s, 594 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186438 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991682 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112025 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131376 s, 79.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77e: check TBF NID nrs policy ============ 21:01:22 (1712278882) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.32@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10438 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00785668 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961487 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269427 s, 38.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367575 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382765 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356296 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307273 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380225 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369689 s, 28.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0448474 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365892 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401471 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344497 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426961 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459916 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348756 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011769 s, 89.1 MB/s , 0.0167053 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105716 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149056 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136791 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013949 s, 75.2 MB/s , 0.0156706 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136158 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203984 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152827 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016655 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184352 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163018 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177359 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170816 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208236 s, 50.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.11969 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116784 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146783 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015234 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0618144 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0720579 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0742554 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0733652 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0775871 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.086586 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0712876 s, 14.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0849554 s, 12.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0887248 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0787834 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0769791 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0750505 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0796788 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105573 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00374148 s, 280 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150272 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124571 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186011 s, 56.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0135871 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143884 s, 72.9 MB/s , 0.0147872 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146624 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166507 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149011 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149659 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017984 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190134 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160873 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206818 s, 50.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.121453 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130813 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158085 s, 66.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659284 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0764669 s, 13.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0692809 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0749012 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0779862 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.074073 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0790766 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0786492 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0836559 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0890225 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.092483 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.092232 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0787461 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0928037 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118602 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149504 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105875 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129564 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015559 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117855 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135941 s, 77.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145986 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013596 s, 77.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0145526 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0155845 s, 67.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0133824 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148444 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153804 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151966 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153085 s, 68.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.134221 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013996 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138521 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0749073 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.061111 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.08351 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0771669 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0864944 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0842636 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0848217 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0908318 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0868568 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0920921 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0901829 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0792905 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0833995 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0889856 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136887 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102677 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159136 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153878 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144146 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127036 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128982 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131139 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146511 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016982 s, 61.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169929 s, 61.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0162686 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164097 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175028 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177083 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175506 s, 59.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77f: check TBF JobID nrs policy ========== 21:01:33 (1712278893) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.145746 s, 115 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142067 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926739 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135373 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216773 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253879 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224293 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374367 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400459 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388251 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0601652 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0612014 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358324 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451205 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428247 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592377 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127164 s, 8.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948185 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00560673 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00209395 s, 501 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00790282 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934153 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892833 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945974 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092466 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112031 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117506 s, 89.2 MB/s , 0.00387738 s, 270 MB/s 1048576 bytes (1.0 MB) copied, 0.00676875 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119125 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011748 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00494223 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00189164 s, 554 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.113539 s, 148 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102496 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161367 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435329 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0560707 s, 18.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0629987 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0698781 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0730408 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0686419 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0726137 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0770181 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0740764 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0660215 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0664933 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0796154 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0835315 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0756393 s, 13.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112017 s, 93.6 MB/s 1+0 records in 1+0 records out , 0.0147925 s, 70.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0131441 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114809 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134634 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134968 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137815 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127126 s, 82.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157802 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139205 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01488 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152882 s, 68.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0160775 s, 65.2 MB/s , 0.0146213 s, 71.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0207013 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014009 s, 74.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.129089 s, 130 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103961 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144399 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0503547 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0523751 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446156 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519829 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633512 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0629374 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0600069 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.060454 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0540853 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0679792 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0723705 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597206 s, 17.6 MB/s , 0.0648933 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0663243 s, 15.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125211 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990177 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102328 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141425 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0122809 s, 85.4 MB/s , 0.0172102 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013527 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147063 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146176 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00293792 s, 357 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167377 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160361 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165193 s, 63.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177745 s, 59.0 MB/s 1048576 bytes (1.0 MB) copied, 0.017868 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200228 s, 52.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.128089 s, 131 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118905 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203896 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191183 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196932 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155806 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255238 s, 41.1 MB/s , 0.00648352 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200334 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213034 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385855 s, 27.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0657979 s, 15.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0631576 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.068679 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0630078 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673314 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0717197 s, 14.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013305 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998649 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143346 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113442 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138449 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124602 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162821 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156881 s, 66.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126432 s, 82.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0128245 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116954 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125271 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138287 s, 75.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121989 s, 86.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0142387 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156904 s, 66.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77g: Change TBF type directly ============ 21:01:43 (1712278903) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 21:01:49 (1712278909) oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg432-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg432-client: oleg432-server: ssh exited with exit code 2 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg432-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg432-client: oleg432-server: ssh exited with exit code 12 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg432-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg432-client: oleg432-server: ssh exited with exit code 12 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg432-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg432-client: oleg432-server: ssh exited with exit code 12 PASS 77h (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77i: Change rank of TBF rule ============= 21:01:55 (1712278915) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77j: check TBF-OPCode NRS policy ========= 21:02:05 (1712278925) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.254198 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101104 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144177 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183802 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381479 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426764 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393549 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532628 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0885696 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0844108 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0742776 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0855032 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0796781 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.26388 s, 4.0 MB/s , 0.265584 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314757 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.302085 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119185 s, 88.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.013621 s, 77.0 MB/s , 0.0121134 s, 86.6 MB/s , 0.0122512 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017578 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146722 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150915 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165257 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168067 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160792 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168564 s, 62.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174337 s, 60.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0176199 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171132 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173149 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187966 s, 55.8 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97339 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4436 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 21:02:59 (1712278979) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22987 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133908 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115783 s, 90.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0277513 s, 37.8 MB/s , 0.0320205 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296632 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298084 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361215 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490929 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161048 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161086 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182974 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185347 s, 5.7 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.192661 s, 5.4 MB/s , 0.187905 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.194552 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.20247 s, 5.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149746 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156613 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016231 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173795 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175758 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015199 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176692 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183618 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177953 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173917 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207262 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175262 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189112 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205701 s, 51.0 MB/s , 0.0195608 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211694 s, 49.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8366 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9603 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22403 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017401 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154457 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219889 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992679 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0322027 s, 32.6 MB/s , 0.0147566 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284841 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021905 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166319 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1665 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.182367 s, 5.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179067 s, 5.9 MB/s , 0.156812 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162225 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188077 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177994 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991754 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108354 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112887 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126991 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116468 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133928 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135909 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140872 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124694 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131304 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134099 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124905 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013138 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130661 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109576 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142786 s, 73.4 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8406 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.947 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (133s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 21:05:13 (1712279113) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.437064 s, 38.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995855 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.0110575 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018955 s, 55.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129031 s, 81.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0208107 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135712 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985609 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169373 s, 61.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.152727 s, 6.9 MB/s , 0.15186 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164879 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15565 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169544 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156678 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181486 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17503 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101833 s, 103 MB/s , 0.0111914 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115222 s, 91.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0124312 s, 84.4 MB/s , 0.00228076 s, 460 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131541 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137545 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135939 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145461 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014346 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0166488 s, 63.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0136777 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151903 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140383 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014621 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165296 s, 63.4 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0298 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9591 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg432-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22104 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020464 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265371 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205996 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145323 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315029 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229226 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034042 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255174 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171701 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17292 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186814 s, 5.6 MB/s , 0.180866 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168241 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162968 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184028 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176142 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010463 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.00978331 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010556 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100133 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116057 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114747 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125701 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113942 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116302 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.01279 s, 82.0 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149374 s, 70.2 MB/s , 0.0134769 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128381 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137336 s, 76.4 MB/s , 0.0146392 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015611 s, 67.2 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0285 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9586 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (134s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 21:07:29 (1712279249) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.251605 s, 66.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014418 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240027 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220446 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326539 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022693 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030951 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381738 s, 27.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0467321 s, 22.4 MB/s , 0.0375189 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382151 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356934 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487195 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411528 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487167 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464653 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495288 s, 21.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010621 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00799791 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827891 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905992 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856792 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959557 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114037 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101935 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102012 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947812 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106257 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110088 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118393 s, 88.6 MB/s , 0.0103668 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109551 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125271 s, 83.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93533 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73457 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.32@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.32@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.223143 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227084 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265294 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565416 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0761672 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0826331 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697011 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0787574 s, 13.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0796585 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0939342 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0734973 s, 14.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0981027 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0831715 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.119854 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117635 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157363 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149529 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225253 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00248748 s, 422 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00228081 s, 460 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00206141 s, 509 MB/s , 0.00216829 s, 484 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00183033 s, 573 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407129 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02589 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197579 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267257 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232666 s, 45.1 MB/s , 0.0459916 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00282396 s, 371 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354183 s, 29.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0356844 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015335 s, 68.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.16494 s, 20.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74299 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.32@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.192619 s, 87.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117244 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141388 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376199 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332414 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398422 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0523176 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0590406 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0608733 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569368 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0615535 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620236 s, 16.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0600039 s, 17.5 MB/s , 0.0636056 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0646761 s, 16.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0668129 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662172 s, 15.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110973 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108658 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091595 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127485 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142425 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015862 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133195 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143437 s, 73.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136745 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126729 s, 82.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0142179 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016696 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148632 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0149388 s, 70.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0151098 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159286 s, 65.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97863 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.88954 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.11425 s, 147 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179931 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198229 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357765 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416306 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434725 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423904 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386031 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428455 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342983 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038228 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343423 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312206 s, 33.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.115705 s, 9.1 MB/s , 0.107067 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105045 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109062 s, 9.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764697 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684043 s, 153 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00826123 s, 127 MB/s , 0.00817082 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101662 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119047 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950668 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891322 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103438 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00988926 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997234 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999939 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102933 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104605 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972152 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128579 s, 81.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78272 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7326 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93993 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73029 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.135704 s, 124 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970558 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311036 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03236 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338383 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309262 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284849 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036278 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329861 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100988 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110726 s, 9.5 MB/s 1048576 bytes (1.0 MB) copied, 0.120888 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125196 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.119023 s, 8.8 MB/s 1+0 records in 1+0 records out , 0.119175 s, 8.8 MB/s 1048576 bytes (1.0 MB) copied, 0.128415 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126641 s, 8.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00777835 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894051 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133224 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111503 s, 94.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0106333 s, 98.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0108627 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0105852 s, 99.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0136512 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127013 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127857 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137595 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134859 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140378 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144495 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145454 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129915 s, 80.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6877 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4321 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.126759 s, 132 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016447 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810781 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960106 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384802 s, 27.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.022458 s, 46.7 MB/s , 0.0304052 s, 34.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0384962 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388418 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034793 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377761 s, 27.8 MB/s , 0.0448424 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336386 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447517 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0632789 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0604441 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0648771 s, 16.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104709 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00790309 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978219 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122201 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140915 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116042 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908269 s, 115 MB/s 1048576 bytes (1.0 MB) copied, 0.0131706 s, 79.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106923 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115222 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117839 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119546 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128762 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137151 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141218 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141619 s, 74.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94384 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73351 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.117022 s, 143 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121999 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114302 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290568 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384765 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383427 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363273 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040303 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404792 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030749 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408146 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426416 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365101 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101717 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104551 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114244 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111294 s, 9.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890366 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985321 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110801 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901124 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105746 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108627 s, 96.5 MB/s , 0.0103019 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112513 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112641 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114935 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108061 s, 97.0 MB/s , 0.0118997 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122435 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0127085 s, 82.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0122409 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011946 s, 87.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79586 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73119 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77507 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73361 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (355s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 21:13:25 (1712279605) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 21:13:30 (1712279610) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg432-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.74343 s, 280 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg432-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0095 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 21:14:21 (1712279661) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0863862 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930972 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124448 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317876 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304604 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322502 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395197 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419756 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390932 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460086 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382909 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423129 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397553 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413703 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483958 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472367 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0525334 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864149 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771568 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111753 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124155 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099376 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120545 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108251 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112446 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106732 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105743 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125428 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109821 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128612 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013402 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130792 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014609 s, 71.8 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg432-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93491 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96773 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.228662 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130682 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117161 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149336 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163287 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00597951 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00563292 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992461 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0242315 s, 43.3 MB/s , 0.0256828 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711294 s, 147 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0497831 s, 21.1 MB/s , 0.0483078 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361162 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423867 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.264778 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.269377 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127251 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113686 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113316 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145058 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123936 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138717 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142062 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141927 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148732 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148767 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014984 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146422 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0154583 s, 67.8 MB/s , 0.015502 s, 67.6 MB/s , 0.0170552 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183784 s, 57.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg432-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg432-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93394 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96736 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77o: Changing rank should not panic ====== 21:15:30 (1712279730) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 21:15:34 (1712279734) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (25s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 21:16:01 (1712279761) mds.MDS.mdt.nrs_policies=tbf oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg432-client: oleg432-server: ssh exited with exit code 36 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 oleg432-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg432-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg432-client: oleg432-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77r: Change type of tbf policy at run time ========================================================== 21:16:12 (1712279772) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 26530 kill md_thread xargs PASS 77r (36s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 78: Enable policy and specify tunings right away ========================================================== 21:16:50 (1712279810) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 79: xattr: intent error ================== 21:16:53 (1712279813) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506679 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 21:16:54.000000000 -0400 Modify: 2024-04-04 21:16:54.000000000 -0400 Change: 2024-04-04 21:16:54.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80a: migrate directory when some children is being opened ========================================================== 21:17:07 (1712279827) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80b: Accessing directory during migration ========================================================== 21:17:09 (1712279829) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81a: rename and stat under striped directory ========================================================== 21:17:12 (1712279832) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 21:17:14 (1712279834) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 21:17:17 (1712279837) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 21:17:19 (1712279839) total: 2500 open/close in 5.26 seconds: 475.65 ops/second snapshot_time 1712279859.297964641 secs.nsecs start_time 1712279840.448622838 secs.nsecs elapsed_time 18.849341803 secs.nsecs open 2505 samples [usecs] 79 3604 889721 343945831 close 2504 samples [usecs] 54 320 215787 19386159 mknod 2501 samples [usecs] 181 3494 757115 251940313 unlink 2500 samples [usecs] 310 6891 1094954 560748976 mkdir 2 samples [usecs] 575 744 1319 884161 rmdir 2 samples [usecs] 1810 2593 4403 9999749 rename 2500 samples [usecs] 266 8421 2596376 3673562436 getattr 3760 samples [usecs] 8 264 59686 1147620 parallel_rename_file 2500 samples [usecs] 266 8421 2596376 3673562436 crossdir_rename 2500 samples [usecs] 266 8421 2596376 3673562436 PASS 81d (21s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 21:17:41 (1712279861) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6742 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6742 PASS 82 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 21:17:45 (1712279865) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 84: 0-nlink race in lu_object_find() ===== 21:17:47 (1712279867) fail_loc=0x8000060b PASS 84 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 85: Lustre API root cache race =========== 21:17:55 (1712279875) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1712279876 Finishing test test1 at 1712279876 PASS 85 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 90: open/create and unlink striped directory ========================================================== 21:17:59 (1712279879) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 91: chmod and unlink striped directory === 21:18:01 (1712279881) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 92: create remote directory under orphan directory ========================================================== 21:18:04 (1712279884) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (0s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 21:18:06 (1712279886) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6067 0x17b3 0x280000400 0 6115 0x17e3 0x240000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6068 0x17b4 0x280000400 0 6116 0x17e4 0x240000400 rc1=0 and rc2=0 PASS 93 (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 94: signal vs CP callback race =========== 21:18:15 (1712279895) osc.lustre-OST0000-osc-ffff8800a9c2d800.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012a4ae000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800a9c2d800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012a4ae000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0356005 s, 230 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00349 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5667: 16695 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0143392 s, 286 kB/s osc.lustre-OST0000-osc-ffff8800a9c2d800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012a4ae000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9c2d800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a4ae000.idle_timeout=debug PASS 94 (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 21:18:27 (1712279907) llite.lustre-ffff8800a9c2d800.fast_read=0 llite.lustre-ffff88012a4ae000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0110714 s, 189 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800a9c2d800.fast_read=1 llite.lustre-ffff88012a4ae000.fast_read=1 PASS 95a (18s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 21:18:47 (1712279927) llite.lustre-ffff8800a9c2d800.fast_read=0 llite.lustre-ffff88012a4ae000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00145263 s, 8.5 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800a9c2d800.fast_read=1 llite.lustre-ffff88012a4ae000.fast_read=1 PASS 95b (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 21:18:56 (1712279936) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 21:18:58 (1712279938) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00143028 s, 2.9 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 21:19:02 (1712279942) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0223272 s, 93.9 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 21:19:05 (1712279945) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0173801 s, 121 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100e: DoM: read on open and file size ==== 21:19:08 (1712279948) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00152486 s, 7.9 MB/s PASS 100e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101a: Discard DoM data on unlink ========= 21:19:11 (1712279951) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000470217 s, 8.7 MB/s PASS 101a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101b: Discard DoM data on rename ========= 21:19:14 (1712279954) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000575177 s, 7.1 MB/s PASS 101b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101c: Discard DoM data on close-unlink === 21:19:17 (1712279957) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000551659 s, 7.4 MB/s PASS 101c (3s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_102 skipping ALWAYS excluded test 102 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 103: Test size correctness with lockahead ========================================================== 21:19:22 (1712279962) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1712279963 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1712279965 Finishing test test23 at 1712279965 PASS 103 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 21:19:30 (1712279970) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 105: Glimpse and lock cancel race ======== 21:19:32 (1712279972) fail_loc=0x416 PASS 105 (27s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106a: Verify the btime via statx() ======= 21:20:01 (1712280001) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106b: Glimpse RPCs test for statx ======== 21:20:03 (1712280003) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106c: Verify statx attributes mask ======= 21:20:05 (1712280005) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107a: Basic grouplock conflict =========== 21:20:08 (1712280008) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0562604 s, 186 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6123 0x17eb 0x240000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 First grouplock blocks second one PASS 107a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 21:20:13 (1712280013) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0550129 s, 191 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6074 0x17ba 0x280000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6742 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 108a: lseek: parallel updates ============ 21:20:22 (1712280022) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00144495 s, 6.9 MB/s llite.lustre-ffff8800a9c2d800.hybrid_io=0 llite.lustre-ffff88012a4ae000.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.13236 s, 2.0 MB/s PASS 108a (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 109: Race with several mount instances on 1 node ========================================================== 21:20:29 (1712280029) 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 3 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 11 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 16 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 17 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 23 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 27 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 35 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 36 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 42 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 47 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions oleg432-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 192.168.204.132@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.204.132@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg432-server' oleg432-server: oleg432-server.virtnet: executing load_modules_local oleg432-server: Loading modules from /home/green/git/lustre-release/lustre oleg432-server: detected 4 online CPUs by sysfs oleg432-server: Force libcfs to create 2 CPU partitions Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre Starting client: oleg432-client.virtnet: -o user_xattr,flock oleg432-server@tcp:/lustre /mnt/lustre2 PASS 109 (414s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 110: do not grant another lock on resend ========================================================== 21:27:25 (1712280445) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-04-04 21:27:25.000000000 -0400 Modify: 2024-04-04 21:27:25.000000000 -0400 Change: 2024-04-04 21:27:25.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg432-client.virtnet oleg432-client.virtnet PASS 110 (64s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 21:28:31 (1712280511) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 112: update max-inherit in default LMV === 21:28:33 (1712280513) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 113: check servers of specified fs ======= 21:28:36 (1712280516) lustre-OST0000-osc-ffff880089759000 active. lustre-OST0001-osc-ffff880089759000 active. lustre-OST0000-osc-ffff880089759000 active. lustre-OST0001-osc-ffff880089759000 active. MGC192.168.204.132@tcp active. PASS 113 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 114: implicit default LMV inherit ======== 21:28:40 (1712280520) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 21:28:43 (1712280523) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) debug_raw_pointers=0 debug_raw_pointers=0 cleanup: ====================================================== == sanityn test complete, duration 5002 sec ============== 21:28:46 (1712280526) === sanityn: start cleanup 21:28:46 (1712280526) === Stopping clients: oleg432-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg432-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 21:29:56 (1712280596) ===