-----============= acceptance-small: sanityn ============----- Fri May 16 08:45:03 EDT 2025 oleg229-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Started clients oleg229-client.virtnet: 192.168.202.129@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg229-client.virtnet: executing check_config_client /mnt/lustre oleg229-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg229-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b4141000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012b32d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b4141000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012b32d000.idle_timeout=debug disable quota as required oleg229-server: oleg229-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378175 s, 27.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6894] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 08:45:23 (1747399523) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (4s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 08:45:27 (1747399527) -rw-r--r-- 1 root root 0 May 16 08:45 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 08:45:30 (1747399530) -rw-r--r-- 1 root root 0 May 16 08:45 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 08:45:33 (1747399533) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 08:45:37 (1747399537) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 08:45:41 (1747399541) total 1 -rwxrwxrwx 1 root root 0 May 16 08:45 f2a -rwxrwxrwx 1 root root 0 May 16 08:45 f2b total 1 -rwxrwxrwx 1 root root 0 May 16 08:45 f2a -rwxrwxrwx 1 root root 0 May 16 08:45 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.0128391 s, 39.9 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 08:45:44 (1747399544) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 08:45:46 (1747399546) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.165358 s, 12.7 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 08:45:49 (1747399549) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 08:45:52 (1747399552) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 08:45:57 (1747399557) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 08:46:00 (1747399560) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 08:46:04 (1747399564) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 08:46:07 (1747399567) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 08:46:10 (1747399570) PASS 9a (4s) == sanityn test 9b: append to striped sparse file ======== 08:46:14 (1747399574) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248984 s, 42.1 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0483602 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 08:46:17 (1747399577) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00704963 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0050354 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00503367 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00434327 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00248466 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00516775 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00457822 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00387716 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00348865 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00341403 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.005842 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00325026 s, 0.3 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 08:46:20 (1747399580) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00656906 s, 468 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.026546 s, 154 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00174461 s, 1.8 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 08:46:23 (1747399583) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 08:46:25 (1747399585) warning: '-runtime' deprecated, use '-t runtime' instead - link 888 (time 1747399596.77 total 10.00 last 88.76) - link 1621 (time 1747399606.78 total 20.02 last 73.21) - link 2448 (time 1747399616.78 total 30.02 last 82.66) - link 3225 (time 1747399626.79 total 40.03 last 77.63) - link 3966 (time 1747399636.80 total 50.04 last 74.04) - link 4865 (time 1747399646.81 total 60.04 last 89.85) - link 5664 (time 1747399656.81 total 70.05 last 79.87) - link 6457 (time 1747399666.81 total 80.05 last 79.28) - link 7572 (time 1747399676.81 total 90.05 last 111.47) total: 8562 link in 100.01 seconds: 85.61 ops/second using seed 1453545893 running for 100 seconds - stat 10000 (time 1747399592 ; total 5 ; last 5) - stat 20000 (time 1747399596 ; total 9 ; last 4) - stat 30000 (time 1747399600 ; total 13 ; last 4) - stat 40000 (time 1747399605 ; total 18 ; last 5) - stat 50000 (time 1747399609 ; total 22 ; last 4) - stat 60000 (time 1747399613 ; total 26 ; last 4) - stat 70000 (time 1747399618 ; total 31 ; last 5) - stat 80000 (time 1747399622 ; total 35 ; last 4) - stat 90000 (time 1747399626 ; total 39 ; last 4) - stat 100000 (time 1747399631 ; total 44 ; last 5) - stat 110000 (time 1747399635 ; total 48 ; last 4) - stat 120000 (time 1747399644 ; total 57 ; last 9) - stat 127320 (time 1747399655 ; total 68 ; last 11) - stat 130000 (time 1747399660 ; total 73 ; last 5) - stat 138441 (time 1747399671 ; total 84 ; last 11) - stat 140000 (time 1747399672 ; total 85 ; last 1) - stat 149053 (time 1747399683 ; total 96 ; last 11) - stat 150000 (time 1747399684 ; total 97 ; last 1) total: 152445 stats in 100 seconds: 1524.449951 stats/second - unlinked 0 (time 1747399637 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile8562) error: No such file or directory total: 8561 unlinks in 86 seconds: 99.546509 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20626) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20628) - No such process PASS 12 (140s) == sanityn test 13: test directory page revocation ======= 08:48:45 (1747399725) f13 f13 PASS 13 (3s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 08:48:48 (1747399728) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4279: 22005 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (3s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 08:48:51 (1747399731) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4279: 22635 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 08:48:54 (1747399734) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23263 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 08:48:56 (1747399736) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23892 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 08:48:58 (1747399738) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24523 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 08:49:01 (1747399741) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7518208 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7518208kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 08:49:13 (1747399753) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.140304 s, 14.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.111962 s, 18.7 MB/s Chance of close/open is 1 in 50 Seed set to 9754 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x510cde 000001[0] 1747399754.993059 trunc from 00000000 to 0x510cdd (0x510cde bytes) 000005[1] 1747399755.033369 mapread 0x250599 thru 0x252665 (0x20cd bytes) 000007[1] 1747399755.068520 mapwrite 0x64d10e thru 0x657042 (0x9f35 bytes) 000008[1] 1747399755.159285 mapread 0x458f23 thru 0x467ae5 (0xebc3 bytes) 000009[1] 1747399755.194424 mapread 0x1c65ba thru 0x1ca259 (0x3ca0 bytes) 000010[1] 1747399755.206043 mapread 0x2e9f5a thru 0x2f73d8 (0xd47f bytes) 000011[1] 1747399755.224963 mapwrite 0x4b5806 thru 0x4b8d3d (0x3538 bytes) truncating to largest ever: 0x735aea 000013[1] 1747399755.270955 trunc from 0x657043 to 0x735ae9 (0xdeaa7 bytes) 000014[1] 1747399755.286162 write 0x82a2fb thru 0x82b729 (0x142f bytes) 000015[0] 1747399755.292040 write 0x2deb93 thru 0x2e7f3b (0x93a9 bytes) 000016[1] 1747399755.309585 mapread 0x12505b thru 0x13374f (0xe6f5 bytes) 000017[0] 1747399755.336471 mapread 0x2f5254 thru 0x2fb925 (0x66d2 bytes) 000019[0] 1747399755.362308 write 0x034adc thru 0x03f3a0 (0xa8c5 bytes) 000021[1] 1747399755.382224 mapwrite 0x105e33 thru 0x10ccd8 (0x6ea6 bytes) 000024[1] 1747399755.463961 read 0x5849d2 thru 0x58c9f2 (0x8021 bytes) 000026[1] 1747399755.486125 write 0x2f8453 thru 0x3025cd (0xa17b bytes) 000027[0] 1747399755.514453 mapread 0x564bc3 thru 0x57218b (0xd5c9 bytes) 000028[1] 1747399755.549935 write 0x89e7dd thru 0x8a1c9a (0x34be bytes) 000029[1] 1747399755.559612 mapread 0x1012b2 thru 0x102487 (0x11d6 bytes) 000032[0] 1747399755.586099 write 0x008f81 thru 0x00b856 (0x28d6 bytes) 000034[0] 1747399755.606561 mapwrite 0x5fb241 thru 0x5fe2db (0x309b bytes) 000035[0] 1747399755.681534 mapwrite 0x29ca5a thru 0x2a950f (0xcab6 bytes) 000036[0] 1747399755.733086 read 0x5a3962 thru 0x5af725 (0xbdc4 bytes) 000037[0] 1747399755.744065 read 0x1b7c9c thru 0x1c4ccc (0xd031 bytes) 000041[0] 1747399755.800991 write 0x4c3eda thru 0x4c8115 (0x423c bytes) 000043[0] 1747399755.809266 mapread 0x80f075 thru 0x815c83 (0x6c0f bytes) 000046[0] 1747399755.816016 read 0x294306 thru 0x2a0054 (0xbd4f bytes) 000047[1] 1747399755.816522 mapread 0x358c05 thru 0x35a1c1 (0x15bd bytes) 000048[1] 1747399755.843769 read 0x22b16b thru 0x23ad83 (0xfc19 bytes) 000049[1] 1747399755.875691 write 0x5ffc51 thru 0x603aa9 (0x3e59 bytes) 000050[0] 1747399755.904347 mapwrite 0x06eadc thru 0x07858c (0x9ab1 bytes) 000051[1] 1747399755.983792 read 0x045125 thru 0x04f735 (0xa611 bytes) 000052[0] 1747399756.037280 write 0x3cf2c1 thru 0x3da23c (0xaf7c bytes) 000053[0] 1747399756.060121 mapwrite 0x5b3b23 thru 0x5b4558 (0xa36 bytes) 000055[0] 1747399756.120362 mapread 0x5ab698 thru 0x5b57ab (0xa114 bytes) 000056[0] 1747399756.161199 read 0x6cd0f0 thru 0x6ce73b (0x164c bytes) 000057[1] 1747399756.175343 mapwrite 0x353f45 thru 0x35684f (0x290b bytes) 000062[0] 1747399756.270143 trunc from 0x8a1c9b to 0x3322ef (0x56f9ab bytes) 000067[1] 1747399756.323083 trunc from 0x3322f0 to 0x2ec342 (0x45fad bytes) 000070[0] 1747399756.369513 mapwrite 0x8c568a thru 0x8d08fb (0xb272 bytes) 000071[0] 1747399756.443341 trunc from 0x8d08fc to 0x0ec9c0 (0x7e3f3b bytes) 000072[1] 1747399756.464556 read 0x061c91 thru 0x062f0d (0x127d bytes) 000073[0] 1747399756.480331 write 0x07cf00 thru 0x07ea4e (0x1b4f bytes) 000077[0] 1747399756.498054 mapwrite 0x8d7371 thru 0x8dad71 (0x3a01 bytes) 000078[1] 1747399756.564428 mapwrite 0x19aa4b thru 0x19d4d5 (0x2a8b bytes) 000079[1] 1747399756.622239 mapread 0x541e1e thru 0x54a5f1 (0x87d4 bytes) 000081[0] 1747399756.685341 trunc from 0x8dad72 to 0x51ad0b (0x3c0066 bytes) truncating to largest ever: 0x80e8a1 000082[1] 1747399756.711811 trunc from 0x51ad0c to 0x80e8a0 (0x2f3b95 bytes) 000084[1] 1747399756.741954 write 0x2fe3c1 thru 0x3024f9 (0x4139 bytes) 000085[0] 1747399756.788596 read 0x150dca thru 0x15bb11 (0xad48 bytes) 000088[0] 1747399756.822702 mapread 0x1e21e3 thru 0x1e9f94 (0x7db2 bytes) 000090[1] 1747399756.836607 mapread 0x0a992f thru 0x0b6fa1 (0xd673 bytes) 000092[1] 1747399756.869960 trunc from 0x80e8a1 to 0x1fc422 (0x61247e bytes) 000095[1] 1747399756.914401 trunc from 0x1fc423 to 0x31b047 (0x11ec25 bytes) 000097[1] 1747399756.938752 mapread 0x15cfde thru 0x161548 (0x456b bytes) 000101[1] 1747399756.959342 mapwrite 0x641095 thru 0x64e54f (0xd4bb bytes) 000105[0] 1747399757.062224 read 0x485b53 thru 0x48e804 (0x8cb2 bytes) 000109[0] 1747399757.119540 mapread 0x021fad thru 0x022476 (0x4ca bytes) 000110[1] 1747399757.130016 write 0x4b6dce thru 0x4c4fcf (0xe202 bytes) 000112[1] 1747399757.146279 read 0x2de01e thru 0x2eae66 (0xce49 bytes) 000114[1] 1747399757.187575 mapread 0x5e8a9b thru 0x5f0c57 (0x81bd bytes) 000116[0] 1747399757.210264 mapread 0x1cb57c thru 0x1d1805 (0x628a bytes) 000117[1] 1747399757.259106 mapwrite 0x0db4c9 thru 0x0df71f (0x4257 bytes) 000118[1] 1747399757.309251 mapread 0x29ec7b thru 0x2a5219 (0x659f bytes) 000119[1] 1747399757.331646 read 0x462fc1 thru 0x46fb3c (0xcb7c bytes) truncating to largest ever: 0x94a33a 000120[0] 1747399757.340221 trunc from 0x64e550 to 0x94a339 (0x2fbdea bytes) 000121[0] 1747399757.371714 trunc from 0x94a33a to 0x09478d (0x8b5bac bytes) 000127[1] 1747399757.403431 read 0x02bf05 thru 0x03a9a6 (0xeaa2 bytes) 000128[0] 1747399757.421750 mapwrite 0x6361f2 thru 0x63651b (0x32a bytes) 000130[1] 1747399757.475799 read 0x358f2b thru 0x368df6 (0xfecc bytes) 000132[0] 1747399757.509073 read 0x440892 thru 0x4444ec (0x3c5b bytes) 000133[1] 1747399757.527963 mapread 0x6110d0 thru 0x6160ec (0x501d bytes) 000134[0] 1747399757.568236 trunc from 0x63651c to 0x1cc537 (0x469fe4 bytes) 000137[0] 1747399757.608183 read 0x058e83 thru 0x061dbb (0x8f39 bytes) 000138[0] 1747399757.631309 mapwrite 0x7137e2 thru 0x71ae88 (0x76a7 bytes) 000140[1] 1747399757.716328 mapread 0x39e590 thru 0x3ab5eb (0xd05c bytes) 000142[0] 1747399757.790614 read 0x614766 thru 0x6157d1 (0x106c bytes) 000143[0] 1747399757.797641 mapread 0x553d47 thru 0x555b85 (0x1e3f bytes) 000147[1] 1747399757.817503 mapwrite 0x46f2f4 thru 0x47bef7 (0xcc04 bytes) 000151[1] 1747399757.919778 mapread 0x451fde thru 0x45648d (0x44b0 bytes) 000155[0] 1747399757.957690 mapwrite 0x3e6280 thru 0x3f5f44 (0xfcc5 bytes) 000157[0] 1747399758.098243 trunc from 0x71ae89 to 0x3029d3 (0x4184b5 bytes) 000158[1] 1747399758.110536 trunc from 0x3029d4 to 0x8a1ae9 (0x59f116 bytes) 000159[0] 1747399758.131919 trunc from 0x8a1aea to 0x9047af (0x62cc6 bytes) 000162[1] 1747399758.180419 read 0x5591bb thru 0x55af2a (0x1d70 bytes) 000163[1] 1747399758.199045 mapread 0x783ca9 thru 0x78d3ee (0x9746 bytes) 000164[1] 1747399758.236950 mapread 0x2e3f15 thru 0x2e99d1 (0x5abd bytes) 000166[1] 1747399758.277907 write 0x3ebbb8 thru 0x3efb69 (0x3fb2 bytes) 000167[1] 1747399758.290236 read 0x2bb455 thru 0x2c7d6e (0xc91a bytes) 000168[1] 1747399758.330168 write 0x8c312c thru 0x8ced7e (0xbc53 bytes) 000169[1] 1747399758.352067 read 0x183238 thru 0x193162 (0xff2b bytes) 000170[1] 1747399758.369196 mapread 0x55d0ec thru 0x55fe57 (0x2d6c bytes) 000171[1] 1747399758.382330 read 0x713a60 thru 0x71d7c2 (0x9d63 bytes) 000172[0] 1747399758.382728 write 0x598d5c thru 0x59970e (0x9b3 bytes) 000177[1] 1747399758.436666 mapread 0x7c9a5c thru 0x7d750f (0xdab4 bytes) 000178[1] 1747399758.476772 read 0x22618d thru 0x22ccda (0x6b4e bytes) 000180[0] 1747399758.477426 mapread 0x61854a thru 0x626e18 (0xe8cf bytes) 000181[0] 1747399758.514409 read 0x817492 thru 0x823ada (0xc649 bytes) 000182[1] 1747399758.520222 mapread 0x13e3cf thru 0x145897 (0x74c9 bytes) 000184[1] 1747399758.542211 mapread 0x071ab5 thru 0x073c23 (0x216f bytes) 000187[1] 1747399758.548411 mapread 0x26723a thru 0x275637 (0xe3fe bytes) 000188[1] 1747399758.577912 mapread 0x056bfb thru 0x065fc5 (0xf3cb bytes) 000190[0] 1747399758.611543 write 0x496ab6 thru 0x4a4f86 (0xe4d1 bytes) 000191[1] 1747399758.621627 write 0x5b0b2f thru 0x5bddd1 (0xd2a3 bytes) 000192[1] 1747399758.641163 trunc from 0x9047b0 to 0x40629f (0x4fe510 bytes) 000193[0] 1747399758.660376 read 0x15fd6b thru 0x1685c3 (0x8859 bytes) 000194[1] 1747399758.673537 write 0x4744b0 thru 0x479661 (0x51b2 bytes) 000195[0] 1747399758.674220 trunc from 0x479662 to 0x050d94 (0x4288cd bytes) 000198[1] 1747399758.703624 read 0x01ee78 thru 0x0275c8 (0x8751 bytes) 000199[1] 1747399758.719054 trunc from 0x050d95 to 0x2946a9 (0x243915 bytes) 000200[1] 1747399758.735334 trunc from 0x2946aa to 0x013b7a (0x280b2f bytes) 000202[0] 1747399758.752531 write 0x0d7791 thru 0x0e1cda (0xa54a bytes) 000203[0] 1747399758.763756 mapread 0x07fbb5 thru 0x083036 (0x3482 bytes) 000205[0] 1747399758.797292 mapwrite 0x0e9a6c thru 0x0f0118 (0x66ad bytes) 000206[1] 1747399758.878159 write 0x0128cc thru 0x01631c (0x3a51 bytes) 000210[0] 1747399758.929367 trunc from 0x0f0119 to 0x2a995f (0x1b9847 bytes) 000212[0] 1747399758.972450 mapwrite 0x5b8deb thru 0x5c193b (0x8b51 bytes) 000214[1] 1747399759.060214 mapread 0x57f0f5 thru 0x58d709 (0xe615 bytes) 000215[0] 1747399759.101990 write 0x934f27 thru 0x940da3 (0xbe7d bytes) 000216[0] 1747399759.117876 write 0x3f45b4 thru 0x3ff80f (0xb25c bytes) 000218[1] 1747399759.131214 read 0x5bf2d7 thru 0x5c9119 (0x9e43 bytes) 000219[0] 1747399759.152617 mapwrite 0x3e3cb9 thru 0x3eabd5 (0x6f1d bytes) 000220[1] 1747399759.200726 write 0x47c2c8 thru 0x485bb9 (0x98f2 bytes) 000221[1] 1747399759.215281 write 0x70e06d thru 0x715b29 (0x7abd bytes) 000226[1] 1747399759.253888 read 0x8f9550 thru 0x904f80 (0xba31 bytes) 000231[0] 1747399759.280518 mapread 0x90c42d thru 0x9186b2 (0xc286 bytes) 000236[1] 1747399759.330954 mapread 0x1d9655 thru 0x1df5d0 (0x5f7c bytes) truncating to largest ever: 0x97951f 000237[0] 1747399759.357380 trunc from 0x940da4 to 0x97951e (0x3877b bytes) 000239[0] 1747399759.391169 read 0x2128d6 thru 0x220656 (0xdd81 bytes) 000243[1] 1747399759.437782 write 0x4021ca thru 0x40aad5 (0x890c bytes) 000244[0] 1747399759.454072 read 0x7106de thru 0x7177e4 (0x7107 bytes) 000245[0] 1747399759.459598 mapwrite 0x2e1a10 thru 0x2eda57 (0xc048 bytes) 000246[0] 1747399759.534474 read 0x4e0b45 thru 0x4e0db3 (0x26f bytes) 000249[1] 1747399759.559972 mapwrite 0x44f8a4 thru 0x45ddec (0xe549 bytes) 000251[0] 1747399759.651612 trunc from 0x97951f to 0x33d55e (0x63bfc0 bytes) 000252[0] 1747399759.665298 mapwrite 0x32e978 thru 0x33404e (0x56d7 bytes) 000253[1] 1747399759.702695 mapread 0x150708 thru 0x15764c (0x6f45 bytes) 000254[1] 1747399759.734262 trunc from 0x33d55f to 0x26d1dc (0xd0382 bytes) 000256[0] 1747399759.777713 mapwrite 0x95b2db thru 0x95c08b (0xdb1 bytes) 000258[0] 1747399759.859781 mapwrite 0x698ce8 thru 0x69ff68 (0x7281 bytes) 000259[0] 1747399759.935558 trunc from 0x95c08c to 0x1d7165 (0x784f26 bytes) truncating to largest ever: 0x9b8081 000261[1] 1747399759.958605 trunc from 0x1d7166 to 0x9b8080 (0x7e0f1b bytes) 000262[0] 1747399759.974001 read 0x456cfd thru 0x45e74a (0x7a4e bytes) 000264[1] 1747399759.987212 read 0x5e497e thru 0x5e49f2 (0x75 bytes) 000265[0] 1747399760.016454 mapread 0x411cad thru 0x4167bf (0x4b13 bytes) 000266[0] 1747399760.036606 mapread 0x04b2e6 thru 0x051209 (0x5f24 bytes) 000268[0] 1747399760.064651 mapread 0x7e7362 thru 0x7f575a (0xe3f9 bytes) 000270[0] 1747399760.117347 mapwrite 0x3e5467 thru 0x3f2ab7 (0xd651 bytes) 000271[1] 1747399760.203914 write 0x228141 thru 0x229cbf (0x1b7f bytes) 000272[0] 1747399760.231107 trunc from 0x9b8081 to 0x274443 (0x743c3d bytes) 000275[1] 1747399760.267946 read 0x06b744 thru 0x06de65 (0x2722 bytes) 000276[1] 1747399760.298986 trunc from 0x274444 to 0x94bae7 (0x6d76a4 bytes) 000277[0] 1747399760.332522 write 0x79606f thru 0x79c8d9 (0x686b bytes) 000281[0] 1747399760.387180 trunc from 0x94bae8 to 0x5d9542 (0x3725a5 bytes) 000282[0] 1747399760.406223 mapread 0x2b57e7 thru 0x2bad87 (0x55a1 bytes) 000283[0] 1747399760.421122 mapread 0x01ed48 thru 0x01feda (0x1193 bytes) 000285[1] 1747399760.431634 trunc from 0x5d9543 to 0x0bf42c (0x51a116 bytes) 000286[0] 1747399760.459407 read 0x08d4fd thru 0x09689a (0x939e bytes) 000289[1] 1747399760.479372 trunc from 0x0bf42d to 0x434f22 (0x375af6 bytes) 000290[1] 1747399760.498327 read 0x3d35d9 thru 0x3d42e9 (0xd11 bytes) 000292[1] 1747399760.513168 mapwrite 0x886d21 thru 0x889c8a (0x2f6a bytes) 000295[1] 1747399760.609508 read 0x6d18c6 thru 0x6df910 (0xe04b bytes) 000296[1] 1747399760.616503 read 0x1d7bfd thru 0x1d9e3c (0x2240 bytes) 000297[1] 1747399760.633757 read 0x65b5d8 thru 0x65cc20 (0x1649 bytes) 000298[1] 1747399760.643977 read 0x5795dc thru 0x57f250 (0x5c75 bytes) 000299[1] 1747399760.649887 mapread 0x2c835e thru 0x2d3c1a (0xb8bd bytes) 000302[1] 1747399760.657999 mapread 0x12cf8e thru 0x139c12 (0xcc85 bytes) 000303[0] 1747399760.705889 write 0x4354aa thru 0x436c6d (0x17c4 bytes) 000304[0] 1747399760.729262 write 0x9bd7b5 thru 0x9c8a46 (0xb292 bytes) 000305[1] 1747399760.739208 read 0x17e1cf thru 0x183944 (0x5776 bytes) 000308[1] 1747399760.764572 write 0x3ff42b thru 0x40c64e (0xd224 bytes) 000312[0] 1747399760.823407 mapread 0x04e8bc thru 0x052937 (0x407c bytes) 000313[0] 1747399760.851815 mapwrite 0x7ac8f2 thru 0x7b4d26 (0x8435 bytes) 000314[1] 1747399760.957572 mapwrite 0x877b99 thru 0x8836cf (0xbb37 bytes) 000315[0] 1747399761.058133 read 0x5bf459 thru 0x5c4350 (0x4ef8 bytes) 000316[0] 1747399761.065797 read 0x2544ed thru 0x25b82f (0x7343 bytes) 000320[1] 1747399761.117372 mapwrite 0x108a86 thru 0x10f8d1 (0x6e4c bytes) 000321[1] 1747399761.222177 read 0x2c456b thru 0x2c83bb (0x3e51 bytes) 000323[1] 1747399761.246002 write 0x0684ec thru 0x06d716 (0x522b bytes) 000325[0] 1747399761.261648 mapwrite 0x6af9b3 thru 0x6b5644 (0x5c92 bytes) 000328[1] 1747399761.372509 read 0x2b337d thru 0x2b5c03 (0x2887 bytes) 000329[0] 1747399761.403046 read 0x4726e3 thru 0x47e51a (0xbe38 bytes) 000330[0] 1747399761.416018 mapwrite 0x2c7fd6 thru 0x2d7042 (0xf06d bytes) 000331[1] 1747399761.537041 write 0x0ad740 thru 0x0b0bf6 (0x34b7 bytes) 000337[1] 1747399761.573894 trunc from 0x9c8a47 to 0x6fea5c (0x2c9fea bytes) 000338[1] 1747399761.597683 mapwrite 0x581044 thru 0x5845ed (0x35aa bytes) 000340[1] 1747399761.643976 trunc from 0x6fea5d to 0x5029e8 (0x1fc074 bytes) 000342[1] 1747399761.676997 read 0x2fb708 thru 0x308305 (0xcbfe bytes) 000345[1] 1747399761.699018 read 0x2a9e4f thru 0x2adcf4 (0x3ea6 bytes) 000347[0] 1747399761.705336 mapwrite 0x2dce5c thru 0x2ec870 (0xfa15 bytes) 000348[1] 1747399761.805482 trunc from 0x5029e9 to 0x707782 (0x204d9a bytes) 000349[0] 1747399761.829671 write 0x8cf5f9 thru 0x8dee5c (0xf864 bytes) 000354[0] 1747399761.885101 write 0x5009e3 thru 0x50388f (0x2ead bytes) 000355[1] 1747399761.916423 mapwrite 0x663ff8 thru 0x66de38 (0x9e41 bytes) 000358[1] 1747399762.005266 read 0x39f9a4 thru 0x3a057b (0xbd8 bytes) 000361[1] 1747399762.078559 write 0x875cf8 thru 0x87e37c (0x8685 bytes) 000362[1] 1747399762.095908 read 0x670213 thru 0x671e5a (0x1c48 bytes) 000363[0] 1747399762.105002 read 0x00a0e2 thru 0x010ad9 (0x69f8 bytes) 000364[0] 1747399762.172055 read 0x6cc097 thru 0x6d948c (0xd3f6 bytes) 000365[1] 1747399762.172449 trunc from 0x8dee5d to 0x12715d (0x7b7cff bytes) 000367[0] 1747399762.209708 write 0x45e0a3 thru 0x461271 (0x31cf bytes) 000368[0] 1747399762.225772 read 0x27f81d thru 0x284c42 (0x5426 bytes) 000369[0] 1747399762.241334 trunc from 0x461272 to 0x65827c (0x1f700b bytes) 000371[1] 1747399762.275286 mapread 0x077bab thru 0x07c82d (0x4c83 bytes) 000374[0] 1747399762.309095 read 0x4247ec thru 0x43161b (0xce30 bytes) 000376[1] 1747399762.324608 write 0x7bc8f4 thru 0x7c3b17 (0x7224 bytes) 000377[1] 1747399762.331099 mapread 0x4842e1 thru 0x48660d (0x232d bytes) 000380[0] 1747399762.342508 trunc from 0x7c3b18 to 0x58bd20 (0x237df7 bytes) 000381[0] 1747399762.375358 mapwrite 0x502715 thru 0x50817d (0x5a69 bytes) 000382[0] 1747399762.435684 mapwrite 0x983117 thru 0x98d66c (0xa556 bytes) 000384[1] 1747399762.524183 read 0x8415b6 thru 0x84dcd8 (0xc723 bytes) 000386[0] 1747399762.556959 read 0x7f17a7 thru 0x7f5561 (0x3dbb bytes) 000387[1] 1747399762.564739 write 0x0934ee thru 0x0a246c (0xef7f bytes) 000389[0] 1747399762.595121 write 0x02067f thru 0x0285e6 (0x7f68 bytes) 000390[0] 1747399762.611109 write 0x51cb84 thru 0x52c6ec (0xfb69 bytes) 000392[0] 1747399762.625487 mapread 0x1118dc thru 0x11cce9 (0xb40e bytes) 000393[1] 1747399762.663277 mapwrite 0x395177 thru 0x39c61b (0x74a5 bytes) 000394[1] 1747399762.764376 trunc from 0x98d66d to 0x63fab5 (0x34dbb7 bytes) 000395[0] 1747399762.809630 mapwrite 0x932773 thru 0x941d52 (0xf5e0 bytes) 000398[0] 1747399762.907806 write 0x2b85cd thru 0x2c4ba8 (0xc5dc bytes) 000400[1] 1747399762.926099 mapread 0x193c26 thru 0x19bf08 (0x82e3 bytes) 000405[0] 1747399762.980290 trunc from 0x941d53 to 0x1cb09e (0x776cb4 bytes) 000407[0] 1747399763.002111 trunc from 0x1cb09f to 0x762f00 (0x597e62 bytes) 000408[0] 1747399763.020682 mapread 0x520e0a thru 0x524065 (0x325c bytes) 000410[0] 1747399763.062997 mapwrite 0x617ceb thru 0x61f096 (0x73ac bytes) 000413[1] 1747399763.148259 mapwrite 0x9a18a9 thru 0x9a24a2 (0xbfa bytes) 000414[1] 1747399763.233246 mapwrite 0x9c2b8f thru 0x9d0a8a (0xdefc bytes) 000417[1] 1747399763.336553 mapwrite 0x4a6aaa thru 0x4b26a7 (0xbbfe bytes) 000419[0] 1747399763.438675 trunc from 0x9d0a8b to 0x742b92 (0x28def8 bytes) 000420[1] 1747399763.471724 write 0x60a278 thru 0x617aa7 (0xd830 bytes) 000421[0] 1747399763.496566 read 0x12be4c thru 0x1384ff (0xc6b4 bytes) 000422[1] 1747399763.511473 write 0x76d9ba thru 0x77bef8 (0xe53f bytes) 000424[1] 1747399763.520649 read 0x2134c4 thru 0x2173a7 (0x3ee4 bytes) 000426[1] 1747399763.551267 write 0x76b952 thru 0x7799e7 (0xe096 bytes) 000428[1] 1747399763.569475 mapread 0x1b3aa0 thru 0x1b8228 (0x4789 bytes) 000429[0] 1747399763.604990 mapwrite 0x8ee1c8 thru 0x8fdfb6 (0xfdef bytes) 000430[1] 1747399763.728687 mapread 0x2bb291 thru 0x2c3294 (0x8004 bytes) 000431[0] 1747399763.783599 mapread 0x724982 thru 0x732f74 (0xe5f3 bytes) 000432[0] 1747399763.784520 read 0x313417 thru 0x31a097 (0x6c81 bytes) 000433[0] 1747399763.785065 mapread 0x7e6f84 thru 0x7f0a9d (0x9b1a bytes) 000435[1] 1747399763.825977 read 0x541dc3 thru 0x5425fc (0x83a bytes) 000436[0] 1747399763.862599 mapwrite 0x85610b thru 0x8625d1 (0xc4c7 bytes) 000437[1] 1747399763.952976 read 0x7fbd34 thru 0x809b35 (0xde02 bytes) 000439[0] 1747399763.994674 mapread 0x738bc8 thru 0x74273f (0x9b78 bytes) 000440[1] 1747399764.017678 write 0x6818f4 thru 0x689578 (0x7c85 bytes) 000441[1] 1747399764.031269 mapwrite 0x21b4f7 thru 0x2212b4 (0x5dbe bytes) 000443[0] 1747399764.085449 trunc from 0x8fdfb7 to 0x0f1bbc (0x80c3fa bytes) 000445[0] 1747399764.125896 write 0x92529b thru 0x933868 (0xe5ce bytes) 000447[0] 1747399764.130763 mapread 0x370fc5 thru 0x3790ca (0x8106 bytes) 000449[0] 1747399764.153126 mapread 0x8c059b thru 0x8ca09a (0x9b00 bytes) 000452[1] 1747399764.199757 mapread 0x4d8a5e thru 0x4e8a33 (0xffd6 bytes) 000453[1] 1747399764.310731 mapread 0x1088eb thru 0x115680 (0xcd96 bytes) 000459[0] 1747399764.365380 mapread 0x6f5c27 thru 0x703c13 (0xdfed bytes) 000463[1] 1747399764.412270 mapread 0x8a4524 thru 0x8b13eb (0xcec8 bytes) 000470[1] 1747399764.447634 mapwrite 0x0261fe thru 0x02b875 (0x5678 bytes) 000471[1] 1747399764.509563 mapwrite 0x38df1c thru 0x399461 (0xb546 bytes) 000474[1] 1747399764.572853 write 0x167794 thru 0x171f07 (0xa774 bytes) 000475[1] 1747399764.580549 trunc from 0x933869 to 0x155cdd (0x7ddb8b bytes) 000477[1] 1747399764.607061 mapread 0x0db1ac thru 0x0e8d29 (0xdb7e bytes) 000478[1] 1747399764.646002 mapwrite 0x3961bd thru 0x39da49 (0x788d bytes) 000479[0] 1747399764.717110 write 0x67cfcf thru 0x686519 (0x954b bytes) 000481[1] 1747399764.744084 trunc from 0x68651a to 0x65c96f (0x29baa bytes) 000482[0] 1747399764.769618 trunc from 0x65c970 to 0x586c9c (0xd5cd3 bytes) 000483[0] 1747399764.799496 write 0x7c0d86 thru 0x7cdd41 (0xcfbc bytes) 000484[1] 1747399764.809611 write 0x4fd836 thru 0x50a863 (0xd02e bytes) 000486[0] 1747399764.829664 read 0x0a0a2d thru 0x0a0c2c (0x200 bytes) 000487[0] 1747399764.866233 mapwrite 0x9c1f94 thru 0x9cf233 (0xd2a0 bytes) 000488[1] 1747399764.990495 mapwrite 0x56948e thru 0x56cd11 (0x3884 bytes) 000490[0] 1747399765.102218 read 0x721b7b thru 0x72a73e (0x8bc4 bytes) 000492[1] 1747399765.158384 mapwrite 0x69da5d thru 0x69e9fa (0xf9e bytes) 000493[1] 1747399765.232934 trunc from 0x9cf234 to 0x42f950 (0x59f8e3 bytes) 000494[0] 1747399765.254055 mapwrite 0x4a2990 thru 0x4ab963 (0x8fd4 bytes) 000495[1] 1747399765.341261 trunc from 0x4ab964 to 0x94cb01 (0x4a119e bytes) 000496[0] 1747399765.362037 write 0x1c14d9 thru 0x1cdc2e (0xc756 bytes) 000497[0] 1747399765.386071 write 0x2c5cd6 thru 0x2d5458 (0xf783 bytes) 000499[0] 1747399765.412302 mapread 0x081a82 thru 0x0853fa (0x3979 bytes) 000500[0] 1747399765.423697 write 0x0622a8 thru 0x06dd2f (0xba88 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9765 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x6846bf 000003[0] 1747399765.757880 trunc from 00000000 to 0x6846be (0x6846bf bytes) 000004[1] 1747399765.776154 mapread 0x4a6000 thru 0x4b0192 (0xa193 bytes) 000006[1] 1747399765.800976 mapread 0x5a0000 thru 0x5a7cac (0x7cad bytes) 000009[1] 1747399765.815393 write 0x91e000 thru 0x926d62 (0x8d63 bytes) 000010[1] 1747399765.822566 mapwrite 0x9e7000 thru 0x9f1ab8 (0xaab9 bytes) 000015[1] 1747399765.937669 mapwrite 0x69e000 thru 0x6aa87f (0xc880 bytes) 000017[0] 1747399766.011457 read 0x263000 thru 0x26614b (0x314c bytes) 000018[1] 1747399766.018687 mapwrite 0x01f000 thru 0x01fd1b (0xd1c bytes) 000020[0] 1747399766.068258 mapwrite 0x51b000 thru 0x5237fc (0x87fd bytes) 000021[1] 1747399766.133814 trunc from 0x9f1ab9 to 0x4a1a11 (0x5500a7 bytes) 000024[1] 1747399766.162757 read 0x0c9000 thru 0x0d8242 (0xf243 bytes) 000028[1] 1747399766.167798 trunc from 0x4a1a12 to 0x0a7f96 (0x3f9a7b bytes) 000030[0] 1747399766.194578 mapwrite 0x351000 thru 0x357778 (0x6779 bytes) 000031[1] 1747399766.257741 write 0x9a0000 thru 0x9aad6c (0xad6d bytes) 000032[0] 1747399766.266909 mapwrite 0x1ff000 thru 0x20c07b (0xd07c bytes) 000034[1] 1747399766.363690 mapread 0x3a1000 thru 0x3a6d22 (0x5d23 bytes) 000036[0] 1747399766.387787 write 0x726000 thru 0x72e8d3 (0x88d4 bytes) 000037[0] 1747399766.396357 mapread 0x5d4000 thru 0x5d640c (0x240d bytes) 000039[0] 1747399766.410297 trunc from 0x9aad6d to 0x2f4bde (0x6b618e bytes) 000040[0] 1747399766.431896 mapread 0x17b000 thru 0x18ad68 (0xfd69 bytes) 000042[1] 1747399766.463818 mapread 0x05e000 thru 0x06223e (0x423f bytes) truncating to largest ever: 0x8330b4 000044[1] 1747399766.484395 trunc from 0x2f4bdf to 0x8330b3 (0x53e4d5 bytes) 000045[1] 1747399766.501574 read 0x803000 thru 0x807070 (0x4071 bytes) 000049[1] 1747399766.546443 write 0x70f000 thru 0x71229d (0x329e bytes) truncating to largest ever: 0x8e70fc 000050[0] 1747399766.551679 trunc from 0x8330b4 to 0x8e70fb (0xb4048 bytes) 000051[0] 1747399766.570164 mapread 0x280000 thru 0x282dd9 (0x2dda bytes) 000053[0] 1747399766.594443 write 0x698000 thru 0x699bed (0x1bee bytes) 000054[0] 1747399766.606333 read 0x395000 thru 0x3a2a7f (0xda80 bytes) 000056[0] 1747399766.642268 read 0x736000 thru 0x73bb36 (0x5b37 bytes) 000057[1] 1747399766.642579 mapwrite 0x41b000 thru 0x4264f8 (0xb4f9 bytes) 000059[0] 1747399766.765717 read 0x76a000 thru 0x77282f (0x8830 bytes) 000060[0] 1747399766.789346 mapread 0x6f2000 thru 0x6fb5af (0x95b0 bytes) 000062[0] 1747399766.809279 read 0x702000 thru 0x708eca (0x6ecb bytes) 000064[1] 1747399766.817209 mapread 0x6db000 thru 0x6e01ac (0x51ad bytes) 000065[1] 1747399766.832391 write 0x842000 thru 0x848c53 (0x6c54 bytes) 000068[0] 1747399766.848661 read 0x8ca000 thru 0x8caff1 (0xff2 bytes) 000069[0] 1747399766.862584 mapread 0x67b000 thru 0x689497 (0xe498 bytes) 000073[1] 1747399766.904611 write 0x701000 thru 0x705ff8 (0x4ff9 bytes) 000074[1] 1747399766.913305 mapread 0x453000 thru 0x458b52 (0x5b53 bytes) 000076[0] 1747399766.944770 read 0x56d000 thru 0x57163a (0x463b bytes) 000077[1] 1747399766.985767 write 0x4aa000 thru 0x4b4598 (0xa599 bytes) 000078[1] 1747399767.007092 trunc from 0x8e70fc to 0x4a61c5 (0x440f36 bytes) 000080[1] 1747399767.034737 mapwrite 0x337000 thru 0x337fcc (0xfcd bytes) 000081[0] 1747399767.071366 write 0x2f2000 thru 0x30110f (0xf110 bytes) 000083[1] 1747399767.094913 mapread 0x13f000 thru 0x14b7c5 (0xc7c6 bytes) 000084[1] 1747399767.155649 mapread 0x2d7000 thru 0x2dfff2 (0x8ff3 bytes) 000085[1] 1747399767.178000 write 0x887000 thru 0x890211 (0x9212 bytes) 000087[0] 1747399767.202202 write 0x66b000 thru 0x66ef6c (0x3f6d bytes) 000089[1] 1747399767.233026 mapread 0x7e7000 thru 0x7f63a6 (0xf3a7 bytes) 000091[0] 1747399767.301227 read 0x061000 thru 0x06aa88 (0x9a89 bytes) 000092[1] 1747399767.349356 trunc from 0x890212 to 0x7387cb (0x157a46 bytes) 000093[0] 1747399767.396258 read 0x462000 thru 0x46ee79 (0xce7a bytes) 000094[1] 1747399767.431114 mapread 0x001000 thru 0x0101ab (0xf1ac bytes) 000095[1] 1747399767.480730 trunc from 0x7387cc to 0x43fb62 (0x2f8c69 bytes) 000096[0] 1747399767.513716 mapread 0x079000 thru 0x07993f (0x940 bytes) 000097[0] 1747399767.530505 write 0x2eb000 thru 0x2f7fdf (0xcfe0 bytes) 000099[0] 1747399767.558135 mapread 0x158000 thru 0x15e4de (0x64df bytes) 000105[1] 1747399767.646557 trunc from 0x43fb63 to 0x0e59bf (0x35a1a3 bytes) 000106[1] 1747399767.678144 mapread 0x0dc000 thru 0x0e5748 (0x9749 bytes) 000107[1] 1747399767.717400 mapwrite 0x930000 thru 0x935d91 (0x5d92 bytes) 000108[1] 1747399767.804534 mapwrite 0x222000 thru 0x22f03c (0xd03d bytes) 000109[0] 1747399767.885652 mapwrite 0x076000 thru 0x079442 (0x3443 bytes) 000110[1] 1747399767.963785 write 0x061000 thru 0x0677f8 (0x67f9 bytes) 000111[1] 1747399767.982189 write 0x957000 thru 0x961c6c (0xac6d bytes) 000112[0] 1747399768.000625 mapwrite 0x6af000 thru 0x6bb4b0 (0xc4b1 bytes) 000113[0] 1747399768.121749 mapwrite 0x397000 thru 0x3a4352 (0xd353 bytes) 000114[1] 1747399768.219731 trunc from 0x961c6d to 0x77eff8 (0x1e2c74 bytes) 000117[1] 1747399768.259965 mapread 0x3d9000 thru 0x3e3d5a (0xad5b bytes) 000119[1] 1747399768.313131 trunc from 0x77eff9 to 0x5bdc4a (0x1c13ae bytes) 000120[1] 1747399768.324184 mapwrite 0x6af000 thru 0x6b6e5f (0x7e60 bytes) 000122[0] 1747399768.419312 mapwrite 0x928000 thru 0x931e4e (0x9e4f bytes) 000125[1] 1747399768.500947 mapread 0x8ec000 thru 0x8f8bac (0xcbad bytes) 000126[1] 1747399768.563144 trunc from 0x931e4f to 0x7b3f8c (0x17dec2 bytes) 000128[0] 1747399768.602173 write 0x0c4000 thru 0x0cc9bb (0x89bc bytes) 000132[0] 1747399768.640502 read 0x375000 thru 0x37fee7 (0xaee8 bytes) 000133[0] 1747399768.678178 mapread 0x487000 thru 0x489c12 (0x2c13 bytes) 000137[1] 1747399768.709333 write 0x5c7000 thru 0x5ce7a4 (0x77a5 bytes) 000138[1] 1747399768.734114 mapread 0x55a000 thru 0x5688ef (0xe8f0 bytes) 000139[0] 1747399768.817796 read 0x556000 thru 0x557b79 (0x1b7a bytes) 000140[0] 1747399768.866217 read 0x01c000 thru 0x01fb6f (0x3b70 bytes) 000141[0] 1747399768.903944 read 0x3ba000 thru 0x3c529a (0xb29b bytes) 000144[0] 1747399768.913607 mapwrite 0x58f000 thru 0x59ecb2 (0xfcb3 bytes) 000148[1] 1747399769.056304 mapread 0x0a0000 thru 0x0ac5ac (0xc5ad bytes) truncating to largest ever: 0x900921 000149[0] 1747399769.108218 trunc from 0x7b3f8d to 0x900920 (0x14c994 bytes) 000153[0] 1747399769.163746 trunc from 0x900921 to 0x82f168 (0xd17b8 bytes) 000154[0] 1747399769.199896 trunc from 0x82f169 to 0x0e437c (0x74adec bytes) 000158[0] 1747399769.234076 write 0x99b000 thru 0x9a1d8d (0x6d8e bytes) 000159[1] 1747399769.243024 mapwrite 0x899000 thru 0x89fe00 (0x6e01 bytes) 000160[0] 1747399769.351974 write 0x187000 thru 0x19266f (0xb670 bytes) 000161[1] 1747399769.366353 mapread 0x35c000 thru 0x35fb0f (0x3b10 bytes) 000162[0] 1747399769.404739 trunc from 0x9a1d8e to 0x0f7462 (0x8aa92b bytes) 000163[0] 1747399769.432140 read 0x024000 thru 0x0320f9 (0xe0fa bytes) 000169[0] 1747399769.478558 mapwrite 0x704000 thru 0x70a2d0 (0x62d1 bytes) 000170[0] 1747399769.591618 mapwrite 0x991000 thru 0x9945a8 (0x35a9 bytes) 000172[1] 1747399769.678038 mapwrite 0x5e3000 thru 0x5e8408 (0x5409 bytes) 000173[0] 1747399769.765800 read_OD 0x37b000 thru 0x386fff (0xc000 bytes) 000174[0] 1747399769.784898 write_OD 0x27a000 thru 0x287fff (0xe000 bytes) 000176[0] 1747399769.835267 trunc from 0x9945a9 to 0x3bf4a6 (0x5d5102 bytes) 000178[0] 1747399769.869461 mapread 0x38b000 thru 0x392f79 (0x7f7a bytes) 000179[0] 1747399769.907746 mapwrite 0x609000 thru 0x615c02 (0xcc03 bytes) 000180[1] 1747399769.998026 mapwrite 0x111000 thru 0x1166f0 (0x56f1 bytes) 000182[1] 1747399770.106411 write 0x8f6000 thru 0x8ff7c5 (0x97c6 bytes) 000183[1] 1747399770.107705 write 0x342000 thru 0x34c9d1 (0xa9d2 bytes) 000187[0] 1747399770.122694 write_OD 0x0ba000 thru 0x0befff (0x5000 bytes) 000188[1] 1747399770.175653 mapread 0x3cf000 thru 0x3cf9fe (0x9ff bytes) 000189[1] 1747399770.182958 mapwrite 0x076000 thru 0x083b49 (0xdb4a bytes) 000190[0] 1747399770.280236 trunc from 0x8ff7c6 to 0x811a98 (0xedd2d bytes) 000193[0] 1747399770.301267 mapwrite 0x23d000 thru 0x244c1b (0x7c1c bytes) 000198[0] 1747399770.379676 trunc from 0x811a99 to 0x350acd (0x4c0fcb bytes) 000199[1] 1747399770.400295 mapwrite 0x6f4000 thru 0x6fa22a (0x622b bytes) 000200[1] 1747399770.488518 mapread 0x6c6000 thru 0x6d0a2b (0xaa2c bytes) 000201[1] 1747399770.535325 write 0x8d2000 thru 0x8d4ba9 (0x2baa bytes) 000203[0] 1747399770.554392 trunc from 0x8d4baa to 0x2011e1 (0x6d39c8 bytes) 000204[0] 1747399770.570879 trunc from 0x2011e2 to 0x14f3cb (0xb1e16 bytes) 000206[0] 1747399770.580518 write_OD 0x309000 thru 0x315fff (0xd000 bytes) 000207[0] 1747399770.618005 read_OD 0x296000 thru 0x29dfff (0x8000 bytes) 000208[0] 1747399770.625538 mapwrite 0x6fd000 thru 0x70a7ad (0xd7ae bytes) 000209[0] 1747399770.722204 read_OD 0x53e000 thru 0x543fff (0x6000 bytes) 000210[0] 1747399770.735720 read_OD 0x16e000 thru 0x170fff (0x3000 bytes) 000214[0] 1747399770.744981 write_OD 0x03a000 thru 0x047fff (0xe000 bytes) 000215[0] 1747399770.775204 mapread 0x42a000 thru 0x42a8a4 (0x8a5 bytes) 000217[1] 1747399770.808302 trunc from 0x70a7ae to 0x2829a8 (0x487e05 bytes) 000219[0] 1747399770.845797 read_OD 0x0ff000 thru 0x10bfff (0xd000 bytes) 000220[1] 1747399770.863715 mapwrite 0x507000 thru 0x50a6d1 (0x36d2 bytes) 000221[1] 1747399770.937307 mapwrite 0x179000 thru 0x1824ab (0x94ac bytes) 000222[1] 1747399771.006152 mapwrite 0x3e2000 thru 0x3e9c85 (0x7c86 bytes) 000223[1] 1747399771.060310 mapwrite 0x95e000 thru 0x9633a8 (0x53a9 bytes) 000226[1] 1747399771.145999 mapread 0x6c8000 thru 0x6d5a77 (0xda78 bytes) 000227[0] 1747399771.188864 write_OD 0x771000 thru 0x779fff (0x9000 bytes) 000228[0] 1747399771.225212 write_OD 0x964000 thru 0x96ffff (0xc000 bytes) 000229[0] 1747399771.256245 write_OD 0x5e0000 thru 0x5e6fff (0x7000 bytes) 000230[1] 1747399771.286512 mapread 0x24b000 thru 0x25643c (0xb43d bytes) 000232[1] 1747399771.333923 read 0x49d000 thru 0x49d848 (0x849 bytes) 000235[0] 1747399771.366804 read_OD 0x3d7000 thru 0x3d8fff (0x2000 bytes) 000236[1] 1747399771.370226 mapwrite 0x63e000 thru 0x640b9a (0x2b9b bytes) 000237[1] 1747399771.445418 read 0x77f000 thru 0x788fde (0x9fdf bytes) 000238[1] 1747399771.457451 mapread 0x464000 thru 0x46abe9 (0x6bea bytes) 000239[0] 1747399771.483683 mapwrite 0x640000 thru 0x64585c (0x585d bytes) 000240[1] 1747399771.570281 write 0x1a1000 thru 0x1b09a2 (0xf9a3 bytes) 000241[1] 1747399771.583350 mapread 0x4d3000 thru 0x4d7d55 (0x4d56 bytes) 000242[1] 1747399771.604145 mapread 0x668000 thru 0x66c527 (0x4528 bytes) 000243[0] 1747399771.617169 mapread 0x505000 thru 0x5104ca (0xb4cb bytes) 000245[0] 1747399771.670807 mapread 0x214000 thru 0x214de6 (0xde7 bytes) 000246[1] 1747399771.674630 read 0x682000 thru 0x691df1 (0xfdf2 bytes) 000247[0] 1747399771.695139 mapread 0x72f000 thru 0x72f78d (0x78e bytes) 000252[1] 1747399771.698664 read 0x7b3000 thru 0x7b8edb (0x5edc bytes) 000254[1] 1747399771.699510 write 0x482000 thru 0x48283b (0x83c bytes) 000255[1] 1747399771.720992 read 0x3b2000 thru 0x3b9e60 (0x7e61 bytes) 000256[0] 1747399771.768413 mapread 0x1b6000 thru 0x1be3b1 (0x83b2 bytes) 000257[0] 1747399771.811561 trunc from 0x970000 to 0x0611b5 (0x90ee4a bytes) 000263[0] 1747399771.874538 trunc from 0x0611b6 to 0x5dfd86 (0x57ebd1 bytes) 000264[1] 1747399771.895593 mapwrite 0x690000 thru 0x6949eb (0x49ec bytes) 000266[1] 1747399771.972256 write 0x8f1000 thru 0x8f1798 (0x799 bytes) 000267[0] 1747399771.974738 mapwrite 0x1d7000 thru 0x1ddf80 (0x6f81 bytes) 000268[0] 1747399772.041265 read_OD 0x3d6000 thru 0x3d8fff (0x3000 bytes) 000270[1] 1747399772.052730 mapread 0x74a000 thru 0x750cab (0x6cac bytes) 000273[0] 1747399772.079454 write_OD 0x661000 thru 0x66cfff (0xc000 bytes) 000275[0] 1747399772.121426 mapread 0x385000 thru 0x392b19 (0xdb1a bytes) 000276[0] 1747399772.205392 trunc from 0x8f1799 to 0x6bc8c3 (0x234ed5 bytes) 000277[0] 1747399772.232316 write_OD 0x3bc000 thru 0x3cafff (0xf000 bytes) 000278[0] 1747399772.274600 mapwrite 0x337000 thru 0x33d17f (0x6180 bytes) 000280[0] 1747399772.359550 read_OD 0x413000 thru 0x41efff (0xc000 bytes) 000281[0] 1747399772.372398 trunc from 0x6bc8c4 to 0x02b785 (0x69113e bytes) 000282[0] 1747399772.389173 mapwrite 0x1eb000 thru 0x1ee7d9 (0x37da bytes) 000285[1] 1747399772.489916 mapread 0x1ad000 thru 0x1bcc7b (0xfc7c bytes) 000286[0] 1747399772.556557 mapwrite 0x22c000 thru 0x235ec9 (0x9eca bytes) 000288[1] 1747399772.650882 mapread 0x1d7000 thru 0x1db75d (0x475e bytes) 000289[0] 1747399772.685702 write 0x398000 thru 0x39fd0e (0x7d0f bytes) 000291[0] 1747399772.708259 write 0x980000 thru 0x983a72 (0x3a73 bytes) 000293[1] 1747399772.721116 mapread 0x190000 thru 0x19a6a7 (0xa6a8 bytes) 000295[0] 1747399772.776421 mapread 0x5fa000 thru 0x5fe032 (0x4033 bytes) 000296[1] 1747399772.788535 write 0x219000 thru 0x226b82 (0xdb83 bytes) 000299[0] 1747399772.826113 mapread 0x71e000 thru 0x72b88a (0xd88b bytes) 000300[1] 1747399772.862123 read 0x42e000 thru 0x435e59 (0x7e5a bytes) 000301[1] 1747399772.902428 mapwrite 0x6e7000 thru 0x6e9dc5 (0x2dc6 bytes) 000302[1] 1747399773.014629 write 0x318000 thru 0x320bd8 (0x8bd9 bytes) 000303[0] 1747399773.035518 mapwrite 0x7f6000 thru 0x7fd2d1 (0x72d2 bytes) 000305[0] 1747399773.115152 read 0x51c000 thru 0x524458 (0x8459 bytes) 000306[1] 1747399773.136402 mapwrite 0x289000 thru 0x29587e (0xc87f bytes) 000308[0] 1747399773.246501 mapwrite 0x981000 thru 0x983540 (0x2541 bytes) 000310[0] 1747399773.308809 read 0x33e000 thru 0x347e71 (0x9e72 bytes) 000311[1] 1747399773.351620 trunc from 0x983a73 to 0x6b7e16 (0x2cbc5c bytes) 000314[1] 1747399773.377239 read 0x3bf000 thru 0x3c332f (0x4330 bytes) 000315[0] 1747399773.415594 trunc from 0x6b7e17 to 0x2f19e7 (0x3c642f bytes) 000318[0] 1747399773.457739 trunc from 0x2f19e8 to 0x69a2ad (0x3a88c6 bytes) 000323[0] 1747399773.497759 read 0x31a000 thru 0x31fb9f (0x5ba0 bytes) 000324[1] 1747399773.520786 write 0x912000 thru 0x917134 (0x5135 bytes) 000326[1] 1747399773.559732 read 0x1b4000 thru 0x1ba065 (0x6066 bytes) 000329[0] 1747399773.611664 trunc from 0x917135 to 0x26974f (0x6ad9e5 bytes) 000332[0] 1747399773.661659 read 0x1f6000 thru 0x200b64 (0xab65 bytes) 000333[0] 1747399773.673762 trunc from 0x269750 to 0x0b2f7d (0x1b67d2 bytes) 000334[1] 1747399773.689371 mapwrite 0x891000 thru 0x891dd0 (0xdd1 bytes) 000335[1] 1747399773.784260 write 0x896000 thru 0x8a2ef7 (0xcef8 bytes) 000336[0] 1747399773.794093 trunc from 0x8a2ef8 to 0x87f58a (0x2396d bytes) 000337[1] 1747399773.828360 read 0x0d7000 thru 0x0dfd5e (0x8d5f bytes) 000338[0] 1747399773.887149 mapwrite 0x034000 thru 0x03a255 (0x6256 bytes) 000341[0] 1747399774.047257 write 0x247000 thru 0x24f658 (0x8659 bytes) 000346[0] 1747399774.068200 write 0x97d000 thru 0x97d4ff (0x500 bytes) 000347[1] 1747399774.068876 write 0x83c000 thru 0x8402cc (0x42cd bytes) 000355[1] 1747399774.154858 trunc from 0x97d500 to 0x1cab5d (0x7b29a2 bytes) 000359[1] 1747399774.212373 mapwrite 0x9fa000 thru 0x9ff3a5 (0x53a6 bytes) 000360[0] 1747399774.288203 trunc from 0x9ff3a6 to 0x681c14 (0x37d791 bytes) 000361[1] 1747399774.305399 read 0x369000 thru 0x3730d7 (0xa0d8 bytes) 000363[1] 1747399774.343384 trunc from 0x681c15 to 0x18a607 (0x4f760d bytes) 000364[0] 1747399774.374279 mapread 0x16b000 thru 0x174a02 (0x9a03 bytes) 000367[1] 1747399774.415971 read 0x14f000 thru 0x1579af (0x89b0 bytes) 000368[1] 1747399774.422074 mapwrite 0x3e7000 thru 0x3f44eb (0xd4ec bytes) 000371[1] 1747399774.513501 read 0x198000 thru 0x1a78c2 (0xf8c3 bytes) 000372[1] 1747399774.523742 read 0x1cf000 thru 0x1cf339 (0x33a bytes) 000373[1] 1747399774.534531 mapwrite 0x305000 thru 0x307a58 (0x2a59 bytes) 000375[0] 1747399774.600708 trunc from 0x3f44ec to 0x2a459b (0x14ff50 bytes) 000376[0] 1747399774.630141 mapread 0x266000 thru 0x273786 (0xd787 bytes) 000377[1] 1747399774.684058 read 0x0f6000 thru 0x0f9fda (0x3fdb bytes) 000379[1] 1747399774.712821 mapwrite 0x7e5000 thru 0x7ec137 (0x7138 bytes) 000380[0] 1747399774.821740 mapwrite 0x768000 thru 0x76b652 (0x3653 bytes) 000381[0] 1747399774.902802 write 0x111000 thru 0x115c4b (0x4c4c bytes) 000388[0] 1747399774.945615 write 0x1a3000 thru 0x1a7e6f (0x4e70 bytes) 000392[0] 1747399774.959314 mapwrite 0x8d0000 thru 0x8dd61e (0xd61f bytes) 000393[1] 1747399775.031762 write 0x255000 thru 0x25571e (0x71f bytes) 000396[1] 1747399775.050902 write 0x9b3000 thru 0x9c056c (0xd56d bytes) 000398[0] 1747399775.052185 mapread 0x715000 thru 0x717b32 (0x2b33 bytes) 000399[0] 1747399775.074150 trunc from 0x9c056d to 0x187109 (0x839463 bytes) 000400[0] 1747399775.089151 write 0x057000 thru 0x05ceb5 (0x5eb6 bytes) 000404[1] 1747399775.115576 mapwrite 0x78b000 thru 0x7959ee (0xa9ef bytes) 000406[1] 1747399775.226162 mapwrite 0x91d000 thru 0x92afcf (0xdfd0 bytes) truncating to largest ever: 0x9e329d 000409[1] 1747399775.342195 trunc from 0x92afd0 to 0x9e329c (0xb82cd bytes) 000412[0] 1747399775.375740 write 0x77b000 thru 0x789368 (0xe369 bytes) 000414[0] 1747399775.392769 read 0x54a000 thru 0x54a43b (0x43c bytes) 000415[1] 1747399775.431090 read 0x438000 thru 0x440448 (0x8449 bytes) 000416[0] 1747399775.482585 mapread 0x7f0000 thru 0x7f1f24 (0x1f25 bytes) 000417[0] 1747399775.499086 trunc from 0x9e329d to 0x9aeaac (0x347f0 bytes) 000419[1] 1747399775.543728 read 0x307000 thru 0x307197 (0x198 bytes) 000421[1] 1747399775.589819 mapwrite 0x081000 thru 0x08f1a1 (0xe1a2 bytes) 000422[1] 1747399775.685620 write 0x3d2000 thru 0x3df39a (0xd39b bytes) 000423[1] 1747399775.701296 mapread 0x913000 thru 0x91ec38 (0xbc39 bytes) 000428[0] 1747399775.763278 read 0x202000 thru 0x20d3dc (0xb3dd bytes) 000429[0] 1747399775.817116 mapwrite 0x2aa000 thru 0x2aac22 (0xc23 bytes) 000430[0] 1747399775.911939 read 0x231000 thru 0x2382d7 (0x72d8 bytes) 000431[1] 1747399775.950457 mapwrite 0x4e0000 thru 0x4e6983 (0x6984 bytes) 000432[0] 1747399776.041278 write 0x6ab000 thru 0x6b5cf9 (0xacfa bytes) 000433[0] 1747399776.062990 write 0x67f000 thru 0x67fe73 (0xe74 bytes) 000435[1] 1747399776.079902 trunc from 0x9aeaad to 0x5e6b5f (0x3c7f4d bytes) 000436[1] 1747399776.102062 mapwrite 0x8ad000 thru 0x8b4c61 (0x7c62 bytes) 000437[1] 1747399776.164582 write 0x152000 thru 0x15b7b0 (0x97b1 bytes) 000438[0] 1747399776.174187 write 0x95f000 thru 0x964e61 (0x5e62 bytes) 000441[0] 1747399776.204913 mapwrite 0x16a000 thru 0x17484d (0xa84e bytes) 000442[0] 1747399776.289547 read 0x2ee000 thru 0x2fbb08 (0xdb09 bytes) 000443[0] 1747399776.335951 mapread 0x1dd000 thru 0x1e784a (0xa84b bytes) 000444[0] 1747399776.381783 trunc from 0x964e62 to 0x595c16 (0x3cf24b bytes) 000445[1] 1747399776.401028 mapwrite 0x936000 thru 0x9443f0 (0xe3f1 bytes) 000446[1] 1747399776.492054 read 0x84e000 thru 0x85121c (0x321d bytes) 000447[1] 1747399776.514992 write 0x9e4000 thru 0x9eb4cb (0x74cc bytes) 000449[0] 1747399776.533419 read 0x1f9000 thru 0x20641a (0xd41b bytes) 000450[0] 1747399776.578364 write 0x752000 thru 0x75a254 (0x8255 bytes) 000454[0] 1747399776.602146 read 0x623000 thru 0x6249b1 (0x19b2 bytes) 000455[1] 1747399776.626465 trunc from 0x9eb4cc to 0x10b444 (0x8e0087 bytes) 000459[0] 1747399776.686053 read 0x00d000 thru 0x01a661 (0xd662 bytes) 000460[0] 1747399776.701283 mapwrite 0x779000 thru 0x780d94 (0x7d95 bytes) 000462[1] 1747399776.791207 mapread 0x48a000 thru 0x493a6b (0x9a6c bytes) 000463[1] 1747399776.829667 mapread 0x0ee000 thru 0x0f125b (0x325c bytes) 000466[0] 1747399776.844977 write 0x722000 thru 0x72ddb5 (0xbdb6 bytes) 000468[0] 1747399776.855174 read 0x262000 thru 0x26f652 (0xd653 bytes) 000470[1] 1747399776.888362 mapwrite 0x558000 thru 0x55827e (0x27f bytes) 000472[1] 1747399776.935395 mapread 0x0e9000 thru 0x0f49e1 (0xb9e2 bytes) 000475[1] 1747399776.979253 trunc from 0x780d95 to 0x55484f (0x22c545 bytes) 000477[1] 1747399776.991010 write 0x402000 thru 0x410d91 (0xed92 bytes) 000479[0] 1747399777.013231 mapwrite 0x505000 thru 0x50f09e (0xa09f bytes) 000481[1] 1747399777.102600 trunc from 0x554850 to 0x6c5543 (0x170cf4 bytes) 000482[1] 1747399777.128772 trunc from 0x6c5544 to 0x2925de (0x432f65 bytes) 000483[1] 1747399777.136716 write 0x204000 thru 0x20b763 (0x7764 bytes) 000484[0] 1747399777.149890 read 0x1ce000 thru 0x1d6138 (0x8139 bytes) 000485[1] 1747399777.168386 trunc from 0x2925df to 0x188fde (0x109600 bytes) 000486[0] 1747399777.185174 write 0x28a000 thru 0x295e02 (0xbe03 bytes) 000487[1] 1747399777.208125 mapread 0x08b000 thru 0x09681c (0xb81d bytes) 000489[0] 1747399777.271813 mapwrite 0x60b000 thru 0x612299 (0x729a bytes) 000491[0] 1747399777.360139 trunc from 0x61229a to 0x529733 (0xe8b66 bytes) 000492[0] 1747399777.371126 read 0x23e000 thru 0x23eb44 (0xb45 bytes) 000493[0] 1747399777.376958 trunc from 0x529734 to 0x034fbb (0x4f4778 bytes) 000495[1] 1747399777.401729 mapwrite 0x373000 thru 0x37c1dd (0x91de bytes) 000497[1] 1747399777.501609 mapread 0x2b0000 thru 0x2b484b (0x484c bytes) All operations completed A-OK! PASS 16a (26s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 08:49:39 (1747399779) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.154698 s, 13.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.131947 s, 15.9 MB/s Chance of close/open is 1 in 50 Seed set to 9780 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read truncating to largest ever: 0x6d1f37 000010[1] 1747399780.441729 trunc from 00000000 to 0x6d1f36 (0x6d1f37 bytes) 000012[0] 1747399780.465723 mapread 0x1f77ed thru 0x1f907f (0x1893 bytes) 000013[1] 1747399780.499345 mapwrite 0x04490a thru 0x045314 (0xa0b bytes) 000014[1] 1747399780.599662 mapwrite 0x9a4962 thru 0x9a4f83 (0x622 bytes) 000015[1] 1747399780.672614 write 0x4e3a06 thru 0x4e5043 (0x163e bytes) 000016[1] 1747399780.684354 read 0x910cca thru 0x912ca2 (0x1fd9 bytes) 000018[1] 1747399780.696324 trunc from 0x9a4f84 to 0x260c6a (0x744319 bytes) 000020[1] 1747399780.727936 read 0x1e5f9d thru 0x1e7df6 (0x1e5a bytes) 000021[0] 1747399780.742063 write 0x7077d5 thru 0x707efd (0x729 bytes) 000022[1] 1747399780.755338 mapwrite 0x011173 thru 0x01206f (0xefd bytes) 000023[0] 1747399780.820289 mapwrite 0x094a27 thru 0x095ae4 (0x10be bytes) 000025[1] 1747399780.880552 mapread 0x1abd55 thru 0x1adaaa (0x1d56 bytes) 000026[0] 1747399780.915188 trunc from 0x707efe to 0x0518dd (0x6b6620 bytes) 000027[1] 1747399780.934405 read 0x04d943 thru 0x04ebc8 (0x1286 bytes) 000028[0] 1747399780.960793 mapread 0x00720b thru 0x00865f (0x1455 bytes) 000031[1] 1747399780.989118 mapread 0x0492a8 thru 0x04ab1b (0x1874 bytes) 000032[0] 1747399781.011456 read 0x01aa72 thru 0x01ba88 (0x1017 bytes) 000033[1] 1747399781.014785 mapwrite 0x3b3486 thru 0x3b497c (0x14f7 bytes) 000035[1] 1747399781.133666 mapread 0x32d369 thru 0x32e05e (0xcf6 bytes) 000037[1] 1747399781.155705 write 0x67874a thru 0x67a3d0 (0x1c87 bytes) 000039[1] 1747399781.168025 read 0x137442 thru 0x139285 (0x1e44 bytes) 000041[0] 1747399781.199312 trunc from 0x67a3d1 to 0x20dd1c (0x46c6b4 bytes) 000044[0] 1747399781.271153 trunc from 0x20dd1d to 0x25a6db (0x4c9bf bytes) 000046[1] 1747399781.300843 trunc from 0x25a6dc to 0x410648 (0x1b5f6d bytes) 000049[0] 1747399781.342245 mapwrite 0x0166a8 thru 0x0175ca (0xf23 bytes) 000050[1] 1747399781.390899 mapread 0x2d874f thru 0x2da1c5 (0x1a77 bytes) truncating to largest ever: 0x965c4d 000054[1] 1747399781.440635 trunc from 0x410649 to 0x965c4c (0x555604 bytes) 000055[0] 1747399781.463402 trunc from 0x965c4d to 0x8a8bf8 (0xbd054 bytes) 000056[0] 1747399781.478888 mapread 0x177328 thru 0x17872e (0x1407 bytes) 000059[1] 1747399781.501989 read 0x6c1723 thru 0x6c21fa (0xad8 bytes) 000060[0] 1747399781.517060 mapwrite 0x3ecfaf thru 0x3eeb3e (0x1b90 bytes) 000061[1] 1747399781.566995 trunc from 0x8a8bf9 to 0x749235 (0x15f9c3 bytes) 000064[1] 1747399781.615848 write 0x7fbaef thru 0x7fca0d (0xf1f bytes) 000065[1] 1747399781.630488 mapwrite 0x7881d4 thru 0x78a16a (0x1f97 bytes) 000066[0] 1747399781.697826 read 0x188802 thru 0x189777 (0xf76 bytes) 000068[0] 1747399781.736280 mapwrite 0x7e5fde thru 0x7e73e6 (0x1409 bytes) 000069[1] 1747399781.835168 mapread 0x4f275c thru 0x4f2ca0 (0x545 bytes) 000070[0] 1747399781.851944 trunc from 0x7fca0e to 0x6eba1e (0x110fef bytes) 000071[0] 1747399781.886439 read 0x0afc6d thru 0x0b19b1 (0x1d45 bytes) 000075[1] 1747399781.986698 mapread 0x17d4ea thru 0x17e21b (0xd32 bytes) 000076[0] 1747399782.019718 trunc from 0x6eba1f to 0x832e38 (0x14741a bytes) 000079[1] 1747399782.059681 mapread 0x2ef36f thru 0x2f0201 (0xe93 bytes) 000080[0] 1747399782.089302 write 0x93dfe5 thru 0x93f8ee (0x190a bytes) 000081[0] 1747399782.103427 trunc from 0x93f8ef to 0x5322bb (0x40d633 bytes) 000083[1] 1747399782.122702 mapwrite 0x621dd6 thru 0x623087 (0x12b2 bytes) 000084[1] 1747399782.227719 write 0x952f22 thru 0x953bbb (0xc9a bytes) 000085[0] 1747399782.243754 mapread 0x32cfd3 thru 0x32d5da (0x608 bytes) 000088[1] 1747399782.278298 mapwrite 0x56c661 thru 0x56db68 (0x1508 bytes) 000090[1] 1747399782.328667 mapwrite 0x9166e2 thru 0x917c12 (0x1531 bytes) 000091[0] 1747399782.366697 write 0x27f5fd thru 0x2811b1 (0x1bb5 bytes) 000092[0] 1747399782.386519 read 0x61b165 thru 0x61c8a5 (0x1741 bytes) truncating to largest ever: 0x9bf1c8 000094[1] 1747399782.409436 trunc from 0x953bbc to 0x9bf1c7 (0x6b60c bytes) 000095[0] 1747399782.431505 mapread 0x65a1ea thru 0x65b060 (0xe77 bytes) 000096[0] 1747399782.458788 write 0x371b9c thru 0x372544 (0x9a9 bytes) 000097[1] 1747399782.480298 trunc from 0x9bf1c8 to 0x4574b3 (0x567d14 bytes) 000098[0] 1747399782.513662 write 0x78b9c8 thru 0x78cdd3 (0x140c bytes) 000100[1] 1747399782.542056 read 0x4a3fed thru 0x4a54c0 (0x14d4 bytes) 000102[0] 1747399782.579946 write 0x723c18 thru 0x7256f3 (0x1adc bytes) 000103[1] 1747399782.601495 trunc from 0x78cdd4 to 0x334ab0 (0x458323 bytes) 000104[1] 1747399782.643408 read 0x15011c thru 0x151888 (0x176d bytes) 000106[1] 1747399782.691107 trunc from 0x334ab1 to 0x83428f (0x4ff7df bytes) 000107[1] 1747399782.713756 read 0x02e448 thru 0x02fb6f (0x1728 bytes) 000109[0] 1747399782.776230 mapwrite 0x139182 thru 0x13a5c8 (0x1447 bytes) 000110[1] 1747399782.850545 write 0x2d3a69 thru 0x2d59ff (0x1f97 bytes) 000113[0] 1747399782.878282 mapread 0x575985 thru 0x57684c (0xec8 bytes) 000118[0] 1747399782.937196 trunc from 0x834290 to 0x4c6075 (0x36e21a bytes) 000119[0] 1747399782.975078 write 0x61db3a thru 0x61f8bc (0x1d83 bytes) 000120[1] 1747399782.978967 write 0x05c327 thru 0x05cf74 (0xc4e bytes) 000121[0] 1747399783.002187 write 0x316187 thru 0x31632a (0x1a4 bytes) 000122[1] 1747399783.014615 read 0x1d5edb thru 0x1d73c3 (0x14e9 bytes) 000125[1] 1747399783.058394 write 0x77c732 thru 0x77e1b2 (0x1a81 bytes) 000130[0] 1747399783.091716 mapread 0x5e165e thru 0x5e2d11 (0x16b4 bytes) 000133[0] 1747399783.130396 mapread 0x11e166 thru 0x11f5b6 (0x1451 bytes) 000134[0] 1747399783.147520 mapwrite 0x65ea9d thru 0x660a19 (0x1f7d bytes) 000136[1] 1747399783.213424 write 0x0c0f18 thru 0x0c13ca (0x4b3 bytes) 000137[0] 1747399783.245149 write 0x8fac6b thru 0x8fc68d (0x1a23 bytes) 000138[1] 1747399783.255299 read 0x075ba9 thru 0x076049 (0x4a1 bytes) 000144[1] 1747399783.317299 mapread 0x49472c thru 0x495001 (0x8d6 bytes) 000145[0] 1747399783.317740 read 0x0325ba thru 0x033121 (0xb68 bytes) 000148[1] 1747399783.381881 mapwrite 0x7d3eed thru 0x7d4973 (0xa87 bytes) 000152[0] 1747399783.463296 write 0x97a025 thru 0x97afbf (0xf9b bytes) 000154[1] 1747399783.478342 mapread 0x690fef thru 0x692ab3 (0x1ac5 bytes) 000155[0] 1747399783.490333 write 0x001645 thru 0x001dad (0x769 bytes) 000157[1] 1747399783.520177 write 0x3c3ba0 thru 0x3c5809 (0x1c6a bytes) 000159[1] 1747399783.548439 read 0x6968ff thru 0x69835b (0x1a5d bytes) 000160[1] 1747399783.582870 mapwrite 0x6e9b37 thru 0x6eaa49 (0xf13 bytes) 000163[1] 1747399783.642981 read 0x0c9755 thru 0x0ca46c (0xd18 bytes) 000164[1] 1747399783.689668 mapwrite 0x395081 thru 0x395583 (0x503 bytes) 000165[1] 1747399783.728432 write 0x63190a thru 0x6320fc (0x7f3 bytes) 000171[1] 1747399783.786758 mapread 0x59df1f thru 0x59e8dc (0x9be bytes) 000172[0] 1747399783.796614 mapread 0x1c5123 thru 0x1c65dd (0x14bb bytes) 000173[1] 1747399783.825636 mapread 0x0e8a5f thru 0x0e93cd (0x96f bytes) 000176[0] 1747399783.847147 mapread 0x0e3741 thru 0x0e4ed0 (0x1790 bytes) 000177[1] 1747399783.873665 write 0x2c174c thru 0x2c343b (0x1cf0 bytes) 000180[0] 1747399783.899722 trunc from 0x97afc0 to 0x4bf51b (0x4bbaa4 bytes) 000182[1] 1747399783.944888 read 0x44553c thru 0x4463e4 (0xea9 bytes) 000184[0] 1747399783.971586 mapread 0x0b47f6 thru 0x0b5d40 (0x154b bytes) 000185[1] 1747399783.977054 read 0x020935 thru 0x020ad5 (0x1a1 bytes) 000186[1] 1747399783.990421 mapwrite 0x2b3f38 thru 0x2b4677 (0x740 bytes) 000187[0] 1747399784.063642 mapwrite 0x6aa775 thru 0x6ab36f (0xbfb bytes) 000188[0] 1747399784.126818 read 0x609701 thru 0x60b07d (0x197d bytes) 000189[0] 1747399784.152907 read 0x2a9b95 thru 0x2ab939 (0x1da5 bytes) 000190[0] 1747399784.174794 mapwrite 0x7533a6 thru 0x754699 (0x12f4 bytes) 000192[1] 1747399784.247449 mapread 0x15d999 thru 0x15e47e (0xae6 bytes) 000194[1] 1747399784.272366 trunc from 0x75469a to 0x458907 (0x2fbd92 bytes) 000197[1] 1747399784.329145 read 0x17a698 thru 0x17b12d (0xa96 bytes) 000198[1] 1747399784.357449 read 0x417a4f thru 0x417c36 (0x1e8 bytes) 000199[1] 1747399784.357774 mapwrite 0x7ea1b3 thru 0x7ec155 (0x1fa3 bytes) 000203[0] 1747399784.432009 trunc from 0x7ec156 to 0x996ffc (0x1aaea7 bytes) 000204[0] 1747399784.459052 trunc from 0x996ffd to 0x474860 (0x52279c bytes) 000205[1] 1747399784.471366 mapwrite 0x9a706d thru 0x9a84e7 (0x147b bytes) 000206[0] 1747399784.548118 read 0x93f6f9 thru 0x941598 (0x1ea0 bytes) 000207[0] 1747399784.566415 mapwrite 0x186598 thru 0x187369 (0xdd2 bytes) 000209[0] 1747399784.633645 write 0x14870d thru 0x14892f (0x223 bytes) 000210[0] 1747399784.642902 mapread 0x0a2417 thru 0x0a35e2 (0x11cc bytes) 000214[1] 1747399784.675549 mapread 0x98a4db thru 0x98a6fa (0x220 bytes) 000216[0] 1747399784.693204 trunc from 0x9a84e8 to 0x72ab62 (0x27d985 bytes) 000218[1] 1747399784.723509 read 0x46319c thru 0x463a57 (0x8bc bytes) 000221[0] 1747399784.743732 mapread 0x5be1da thru 0x5be3b4 (0x1db bytes) 000224[1] 1747399784.766962 trunc from 0x72ab63 to 0x725532 (0x5630 bytes) 000225[0] 1747399784.785662 trunc from 0x725533 to 0x4cc6af (0x258e83 bytes) 000226[0] 1747399784.805510 read 0x3947d4 thru 0x396522 (0x1d4f bytes) 000229[1] 1747399784.835623 read 0x3a2416 thru 0x3a3a45 (0x1630 bytes) 000232[1] 1747399784.862691 trunc from 0x4cc6b0 to 0x81b297 (0x34ebe8 bytes) 000234[0] 1747399784.912955 mapread 0x31aab9 thru 0x31ab14 (0x5c bytes) 000235[1] 1747399784.934502 trunc from 0x81b298 to 0x6287ea (0x1f2aad bytes) 000236[1] 1747399784.960936 read 0x483942 thru 0x4857bc (0x1e7b bytes) 000238[0] 1747399784.974437 trunc from 0x6287eb to 0x3005c2 (0x328228 bytes) 000240[0] 1747399785.020305 mapwrite 0x79895d thru 0x7994f1 (0xb95 bytes) 000243[0] 1747399785.099050 mapwrite 0x6a482e thru 0x6a56f8 (0xecb bytes) 000244[1] 1747399785.145813 mapwrite 0x01fd98 thru 0x021a44 (0x1cad bytes) 000245[1] 1747399785.230773 write 0x451aa2 thru 0x4520a7 (0x606 bytes) 000248[1] 1747399785.262680 mapread 0x7466c4 thru 0x7471ee (0xb2b bytes) 000250[0] 1747399785.285905 mapwrite 0x012334 thru 0x013bff (0x18cc bytes) 000251[1] 1747399785.339091 trunc from 0x7994f2 to 0x1a1a56 (0x5f7a9b bytes) 000253[0] 1747399785.371317 mapwrite 0x3dab6f thru 0x3dbdc3 (0x1255 bytes) 000254[1] 1747399785.441173 write 0x2b2bed thru 0x2b3769 (0xb7d bytes) 000255[0] 1747399785.459020 write 0x81dcb4 thru 0x81e901 (0xc4e bytes) 000257[1] 1747399785.491203 read 0x2b956f thru 0x2b96dc (0x16e bytes) 000258[0] 1747399785.519511 read 0x5a3825 thru 0x5a4a3d (0x1219 bytes) 000259[1] 1747399785.546453 write 0x5f417a thru 0x5f5073 (0xefa bytes) 000261[1] 1747399785.592635 write 0x9f0b22 thru 0x9f1423 (0x902 bytes) 000263[1] 1747399785.605117 read 0x5ac709 thru 0x5aca22 (0x31a bytes) 000266[0] 1747399785.639535 mapwrite 0x09ee72 thru 0x09fde2 (0xf71 bytes) 000267[0] 1747399785.702092 read 0x91b485 thru 0x91bea6 (0xa22 bytes) 000269[0] 1747399785.742667 mapwrite 0x8f9f16 thru 0x8fb0c1 (0x11ac bytes) 000270[1] 1747399785.793982 mapread 0x68d27d thru 0x68d44a (0x1ce bytes) 000271[0] 1747399785.811336 mapread 0x47b60c thru 0x47ba2d (0x422 bytes) 000272[1] 1747399785.814384 mapwrite 0x8158a8 thru 0x8165da (0xd33 bytes) 000273[0] 1747399785.874769 read 0x41adaf thru 0x41c63d (0x188f bytes) 000274[1] 1747399785.904358 mapwrite 0x0d1389 thru 0x0d162a (0x2a2 bytes) 000275[1] 1747399785.968451 write 0x822f57 thru 0x8243cd (0x1477 bytes) 000277[1] 1747399785.987614 mapwrite 0x0d1eca thru 0x0d343b (0x1572 bytes) 000278[0] 1747399786.034456 read 0x4629e9 thru 0x4635a1 (0xbb9 bytes) 000279[0] 1747399786.060280 mapread 0x39a5bd thru 0x39a6db (0x11f bytes) 000280[1] 1747399786.069442 read 0x4f4e82 thru 0x4f6904 (0x1a83 bytes) 000283[1] 1747399786.105937 mapwrite 0x9471cd thru 0x9487ad (0x15e1 bytes) 000286[1] 1747399786.188130 mapread 0x2292c9 thru 0x22b258 (0x1f90 bytes) 000287[0] 1747399786.204587 mapread 0x8c0a13 thru 0x8c1146 (0x734 bytes) 000288[1] 1747399786.209620 read 0x863256 thru 0x864cba (0x1a65 bytes) 000289[1] 1747399786.218648 mapwrite 0x5ffb53 thru 0x600ae1 (0xf8f bytes) 000293[1] 1747399786.320909 mapread 0x85bf23 thru 0x85d86a (0x1948 bytes) 000294[1] 1747399786.342293 write 0x3d0909 thru 0x3d21b4 (0x18ac bytes) 000295[0] 1747399786.350412 mapread 0x5313e0 thru 0x53158d (0x1ae bytes) 000296[0] 1747399786.378139 write 0x9bae3f thru 0x9bbef1 (0x10b3 bytes) 000299[1] 1747399786.415606 mapwrite 0x76dc34 thru 0x76e086 (0x453 bytes) 000301[1] 1747399786.484333 mapwrite 0x8cf234 thru 0x8cf443 (0x210 bytes) 000306[0] 1747399786.529952 trunc from 0x9f1424 to 0x953cc4 (0x9d75f bytes) 000307[1] 1747399786.550020 mapread 0x3df736 thru 0x3dfbeb (0x4b6 bytes) 000309[1] 1747399786.569566 trunc from 0x953cc5 to 0x3f1047 (0x562c7d bytes) 000310[1] 1747399786.593977 read 0x37d3de thru 0x37e258 (0xe7b bytes) 000311[1] 1747399786.604927 read 0x0095c8 thru 0x00aade (0x1517 bytes) 000312[1] 1747399786.640646 mapwrite 0x4b50e3 thru 0x4b63b3 (0x12d1 bytes) 000313[1] 1747399786.705267 mapwrite 0x1aa5ed thru 0x1ac4df (0x1ef3 bytes) 000315[0] 1747399786.779923 read 0x043f84 thru 0x0453f0 (0x146d bytes) 000316[0] 1747399786.807714 trunc from 0x4b63b4 to 0x2aa2cd (0x20c0e6 bytes) 000317[1] 1747399786.858886 read 0x0a6bc8 thru 0x0a83b6 (0x17ef bytes) 000321[0] 1747399786.903622 mapwrite 0x6c74ae thru 0x6c8e41 (0x1994 bytes) 000322[0] 1747399787.013681 read 0x5d4e70 thru 0x5d62f6 (0x1487 bytes) 000323[0] 1747399787.033768 mapread 0x1aeddd thru 0x1af943 (0xb67 bytes) 000324[1] 1747399787.043653 write 0x52050c thru 0x520c64 (0x759 bytes) 000328[0] 1747399787.118257 mapwrite 0x5bfc51 thru 0x5c03d8 (0x788 bytes) 000329[0] 1747399787.186721 mapread 0x3195c1 thru 0x319aa9 (0x4e9 bytes) 000332[1] 1747399787.189935 mapwrite 0x12d0cf thru 0x12d652 (0x584 bytes) 000335[1] 1747399787.286292 write 0x2587ce thru 0x258dfe (0x631 bytes) 000338[1] 1747399787.344645 trunc from 0x6c8e42 to 0x1882ae (0x540b93 bytes) 000339[1] 1747399787.363373 read 0x06ab55 thru 0x06b4dd (0x989 bytes) 000340[1] 1747399787.382596 trunc from 0x1882af to 0x6d67ad (0x54e4ff bytes) 000343[0] 1747399787.404690 mapread 0x45a1e0 thru 0x45c14c (0x1f6d bytes) 000344[1] 1747399787.432571 mapread 0x488af9 thru 0x488dfe (0x306 bytes) 000345[0] 1747399787.475581 mapwrite 0x18fb15 thru 0x19029e (0x78a bytes) 000346[1] 1747399787.549165 read 0x29f54a thru 0x29fc0b (0x6c2 bytes) 000347[1] 1747399787.549527 trunc from 0x6d67ae to 0x32b717 (0x3ab096 bytes) 000348[1] 1747399787.579328 mapread 0x2909ce thru 0x2929a6 (0x1fd9 bytes) 000349[1] 1747399787.598914 write 0x4dd7b5 thru 0x4dec77 (0x14c3 bytes) 000350[1] 1747399787.611450 write 0x7e3de8 thru 0x7e47cc (0x9e5 bytes) 000353[1] 1747399787.622026 write 0x7ffff6 thru 0x801049 (0x1054 bytes) 000356[0] 1747399787.642205 trunc from 0x80104a to 0x11d3d8 (0x6e3c71 bytes) 000357[1] 1747399787.680587 write 0x879d95 thru 0x87a8f0 (0xb5c bytes) 000359[0] 1747399787.699769 mapread 0x357159 thru 0x357d1f (0xbc7 bytes) 000362[0] 1747399787.733173 trunc from 0x87a8f1 to 0x1c99f7 (0x6b0ef9 bytes) 000364[0] 1747399787.774365 mapread 0x039ed7 thru 0x03bb83 (0x1cad bytes) 000366[1] 1747399787.795694 write 0x598fa5 thru 0x599acf (0xb2b bytes) 000367[0] 1747399787.808601 mapwrite 0x026673 thru 0x026e87 (0x815 bytes) 000368[0] 1747399787.866541 trunc from 0x599ad0 to 0x7f35f3 (0x259b24 bytes) 000373[0] 1747399787.927352 mapread 0x3ff6a3 thru 0x4006b6 (0x1014 bytes) 000376[1] 1747399787.949608 mapread 0x7f045d thru 0x7f1a97 (0x163b bytes) 000378[1] 1747399787.974998 mapwrite 0x502880 thru 0x503343 (0xac4 bytes) 000379[1] 1747399788.038572 mapread 0x3064c2 thru 0x307592 (0x10d1 bytes) 000381[0] 1747399788.072085 mapread 0x041b1a thru 0x041c02 (0xe9 bytes) 000384[1] 1747399788.111303 mapwrite 0x9e4a24 thru 0x9e6573 (0x1b50 bytes) 000387[0] 1747399788.191700 mapread 0x24caa2 thru 0x24d042 (0x5a1 bytes) 000388[1] 1747399788.221482 mapwrite 0x5f2a43 thru 0x5f40aa (0x1668 bytes) 000389[0] 1747399788.267440 trunc from 0x9e6574 to 0x39b7c1 (0x64adb2 bytes) 000390[0] 1747399788.311722 mapread 0x31e8ab thru 0x31ff94 (0x16ea bytes) 000391[1] 1747399788.329944 write 0x0e7267 thru 0x0e7752 (0x4ec bytes) 000392[0] 1747399788.347143 read 0x230e3f thru 0x231b8f (0xd51 bytes) 000397[0] 1747399788.385634 write 0x58f2bb thru 0x590d2a (0x1a70 bytes) 000399[1] 1747399788.402636 mapwrite 0x3e29a8 thru 0x3e2d30 (0x389 bytes) 000400[1] 1747399788.458880 read 0x33f28b thru 0x33fcb8 (0xa2e bytes) 000401[1] 1747399788.487079 trunc from 0x590d2b to 0x6dcf93 (0x14c269 bytes) 000402[0] 1747399788.522624 mapwrite 0x04dac1 thru 0x04e646 (0xb86 bytes) 000403[0] 1747399788.572390 read 0x05d941 thru 0x05e8a7 (0xf67 bytes) 000406[1] 1747399788.609488 mapwrite 0x89a952 thru 0x89c271 (0x1920 bytes) 000410[1] 1747399788.731365 write 0x91bb3a thru 0x91d4c8 (0x198f bytes) 000412[0] 1747399788.740369 mapwrite 0x5c3d48 thru 0x5c4089 (0x342 bytes) 000413[0] 1747399788.821896 mapwrite 0x06b42f thru 0x06c7e7 (0x13b9 bytes) 000415[1] 1747399788.950454 write 0x6f6fbe thru 0x6f7a3b (0xa7e bytes) 000416[0] 1747399788.988397 write 0x5c669b thru 0x5c66d9 (0x3f bytes) 000417[0] 1747399789.000784 write 0x523b60 thru 0x524c9c (0x113d bytes) 000419[0] 1747399789.022968 mapread 0x73824a thru 0x738dbb (0xb72 bytes) 000421[0] 1747399789.046164 mapread 0x4b6ed4 thru 0x4b8660 (0x178d bytes) 000424[1] 1747399789.100925 read 0x01b58e thru 0x01bb87 (0x5fa bytes) 000426[1] 1747399789.125030 mapread 0x47a5c6 thru 0x47b75f (0x119a bytes) 000427[0] 1747399789.145207 trunc from 0x91d4c9 to 0x224788 (0x6f8d40 bytes) 000428[0] 1747399789.183018 mapwrite 0x17c7ed thru 0x17ca8e (0x2a2 bytes) 000430[1] 1747399789.224695 write 0x66eee3 thru 0x66fc44 (0xd62 bytes) 000431[1] 1747399789.245664 write 0x84c60c thru 0x84d053 (0xa48 bytes) 000432[0] 1747399789.256213 mapread 0x3744ab thru 0x3758cb (0x1421 bytes) 000433[0] 1747399789.291377 read 0x29c453 thru 0x29ce24 (0x9d2 bytes) 000435[0] 1747399789.356816 write 0x2b98c6 thru 0x2ba0b4 (0x7ef bytes) 000436[0] 1747399789.374726 trunc from 0x84d054 to 0x3b6070 (0x496fe3 bytes) 000441[1] 1747399789.406764 mapread 0x045646 thru 0x047119 (0x1ad4 bytes) 000443[1] 1747399789.452211 trunc from 0x3b6071 to 0x8ced6f (0x518cff bytes) 000444[1] 1747399789.475740 mapwrite 0x1dc8f0 thru 0x1ddb98 (0x12a9 bytes) 000445[1] 1747399789.574525 write 0x8dbcea thru 0x8ddc74 (0x1f8b bytes) 000448[0] 1747399789.590662 write 0x28ace7 thru 0x28b765 (0xa7f bytes) 000451[0] 1747399789.647555 trunc from 0x8ddc75 to 0x21a46a (0x6c380a bytes) 000453[0] 1747399789.674383 mapwrite 0x04b987 thru 0x04c9cb (0x1045 bytes) 000461[0] 1747399789.754083 mapwrite 0x4868d9 thru 0x487590 (0xcb8 bytes) 000462[0] 1747399789.812332 mapwrite 0x0e0199 thru 0x0e1226 (0x108e bytes) 000463[0] 1747399789.854857 trunc from 0x487591 to 0x42f43d (0x58153 bytes) truncating to largest ever: 0x9e5535 000464[0] 1747399789.864940 trunc from 0x42f43e to 0x9e5534 (0x5b60f7 bytes) 000467[0] 1747399789.874660 write 0x61a786 thru 0x61c148 (0x19c3 bytes) 000469[0] 1747399789.887876 write 0x7ddc5b thru 0x7de8d4 (0xc7a bytes) 000472[0] 1747399789.909174 write 0x0eb826 thru 0x0ecd99 (0x1574 bytes) 000476[0] 1747399789.929814 read 0x04c451 thru 0x04ca76 (0x626 bytes) 000480[1] 1747399789.949013 mapread 0x74ec3a thru 0x750804 (0x1bcb bytes) 000483[1] 1747399790.004714 mapread 0x8a86ca thru 0x8aa2d7 (0x1c0e bytes) 000484[1] 1747399790.053049 trunc from 0x9e5535 to 0x7e04c1 (0x205073 bytes) 000486[1] 1747399790.098404 mapread 0x77a387 thru 0x77a908 (0x582 bytes) 000487[1] 1747399790.112304 write 0x6724e0 thru 0x67437b (0x1e9c bytes) 000489[1] 1747399790.141942 mapread 0x058dad thru 0x059f22 (0x1176 bytes) 000490[0] 1747399790.157210 trunc from 0x7e04c2 to 0x6d84fb (0x107fc6 bytes) 000491[0] 1747399790.186331 mapread 0x42f773 thru 0x431153 (0x19e1 bytes) 000493[1] 1747399790.229109 read 0x4c7d36 thru 0x4c825c (0x527 bytes) 000495[1] 1747399790.264444 mapread 0x376ac3 thru 0x3781ee (0x172c bytes) 000497[0] 1747399790.315281 mapread 0x30230e thru 0x3042ae (0x1fa1 bytes) 000498[0] 1747399790.326088 mapwrite 0x1abe9b thru 0x1ad437 (0x159d bytes) 000499[0] 1747399790.386380 read 0x45d3e0 thru 0x45da45 (0x666 bytes) 000500[0] 1747399790.409323 trunc from 0x6d84fc to 0x21b9fc (0x4bcaff bytes) All operations completed A-OK! PASS 16b (13s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 08:49:52 (1747399792) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 08:49:53 (1747399793) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.161204 s, 13.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.112421 s, 18.7 MB/s ldlm.namespaces.MGC192.168.202.129@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b4141000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012b32d000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012b32d000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012b32d000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.72579 s, 18.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.22909 s, 85.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.649608 s, 161 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.04915 s, 51.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.78618 s, 27.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.03524 s, 101 MB/s PASS 16d (19s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 08:50:12 (1747399812) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0836294 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0926743 s, 11.3 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 08:50:15 (1747399815) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 08:50:37 (1747399837) TIMEOUT PASS 16g (23s) == sanityn test 16i: read after truncate file ============ 08:51:01 (1747399861) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 08:51:03 (1747399863) fail_loc=0x8000030a PASS 17 (6s) == sanityn test 18: mmap sanity check =========================================================================================== 08:51:09 (1747399869) mmap test1: basic mmap operation (PASS, 0.034735s) mmap test2: MAP_PRIVATE not write back (PASS, 0.020789s) mmap test3: concurrent mmap ops on two nodes (PASS, 10.27s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.2277s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.10489s) mmap test6: check mmap write/read content on two nodes (PASS, 0.13729s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0391s) PASS 18 (17s) == sanityn test 19: test concurrent uncached read races ========================================================================= 08:51:26 (1747399886) oleg229-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg229-client: oleg229-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 08:51:28 (1747399888) PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 08:51:31 (1747399891) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 08:51:33 (1747399893) now is 1747399894 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 new atime is 1747399955 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 08:52:37 (1747399957) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 14336 7524352 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 14336 7524352 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 376 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 14.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117976 376 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235633 369 235264 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 14336 7524352 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117976 376 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235633 369 235264 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117976 376 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235633 369 235264 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 14336 7524352 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 14336 7524352 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 08:52:40 (1747399960) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 08:52:42 (1747399962) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (4s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 08:52:46 (1747399966) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 08:52:47 (1747399967) utime: good mknod times 1747399967 <= 1747399968 <= 1747399968 for /mnt/lustre/f26a utime: good mknod times 1747399967 <= 1747399968 <= 1747399968 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (4s) == sanityn test 26b: sync mtime between ost and mds ====== 08:52:51 (1747399971) PASS 26b (5s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 08:52:58 (1747399978) 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 08:53:02 (1747399982) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 08:53:05 (1747399985) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg229-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.129@tcp:/lustre 7542784 14336 7524352 1% /mnt/lustre2 PASS 31b (14s) == sanityn test 31r: open-rename(replace) race =========== 08:53:19 (1747399999) -rw-r--r-- 1 root root 0 May 16 08:53 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (6s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 08:53:26 (1747400006) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 08:53:27 (1747400007) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 08:53:28 (1747400008) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 08:53:30 (1747400010) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 08:53:31 (1747400011) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b4141000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012b32d000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b4141000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88012b32d000.ost_server_uuid in IDLE state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b4141000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012b32d000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b4141000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012b32d000.ost_server_uuid in FULL state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b4141000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012b32d000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b4141000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012b32d000.ost_server_uuid in IDLE state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b4141000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012b32d000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b4141000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88012b32d000.ost_server_uuid in FULL state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b4141000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012b32d000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b4141000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012b32d000.ost_server_uuid in IDLE state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b4141000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012b32d000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b4141000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88012b32d000.ost_server_uuid in FULL state after 0 sec PASS 34 (83s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 08:54:54 (1747400094) fail_loc=0x80000317 Race attempt 0 Wait for 15682 15824 for 60 sec... - open/close 1892 (time 1747400107.13 total 10.00 last 189.15) - open/close 3810 (time 1747400117.13 total 20.01 last 191.73) total: 4000 open/close in 20.88 seconds: 191.54 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.129@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre 192.168.202.129@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre2 PASS 35 (65s) == sanityn test 36: handle ESTALE/open-unlink correctly == 08:55:59 (1747400159) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg229-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.436518 s, 120 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7523328) after_dd(7471104) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.349914 s, 150 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7523328) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.345818 s, 152 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7520256) after_dd(7471104) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.42391 s, 124 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7523328) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.330394 s, 159 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7522304) after_dd(7467008) after(7518208) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.350951 s, 149 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7518208) after_dd(7464960) after(7516160) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.370232 s, 142 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7516160) after_dd(7467008) after(7518208) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.337456 s, 155 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7518208) after_dd(7467008) after(7518208) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.416408 s, 126 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7518208) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.327632 s, 160 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7520256) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.345156 s, 152 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7522304) after_dd(7467008) after(7518208) PASS 36 (230s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 08:59:49 (1747400389) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 - create 3785 (time 1747400400.92 total 10.00 last 378.49) - create 8012 (time 1747400410.92 total 20.00 last 422.62) total: 10000 create in 24.84 seconds: 402.52 ops/second PASS 37 (29s) == sanityn test 39a: test from 11063 ============================================================================================ 09:00:18 (1747400418) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 09:00:20 (1747400420) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 09:00:23 (1747400423) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 09:00:27 (1747400427) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 09:00:29 (1747400429) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 09:00:38 (1747400438) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 09:00:46 (1747400446) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 09:00:54 (1747400454) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 09:01:01 (1747400461) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 09:01:09 (1747400469) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 09:01:14 (1747400474) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 09:01:19 (1747400479) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 09:01:24 (1747400484) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 09:01:29 (1747400489) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 09:01:34 (1747400494) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 09:01:38 (1747400498) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 09:01:43 (1747400503) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 09:01:47 (1747400507) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (679s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 09:13:06 (1747401186) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 09:13:10 (1747401190) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 09:13:14 (1747401194) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 09:13:19 (1747401199) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 09:13:23 (1747401203) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 09:13:27 (1747401207) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 09:13:31 (1747401211) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 09:13:36 (1747401216) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 09:13:40 (1747401220) PASS 43a (12s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 09:13:52 (1747401232) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 09:13:56 (1747401236) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 09:14:00 (1747401240) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 09:14:04 (1747401244) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 09:14:08 (1747401248) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 09:14:12 (1747401252) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 09:14:17 (1747401257) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 09:14:21 (1747401261) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (0s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 09:14:21 (1747401261) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (33s) == sanityn test 43k: unlink vs create ==================== 09:14:54 (1747401294) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (377s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 09:21:11 (1747401671) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri May 16 09:21:14 EDT 2025 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 09:21:16 (1747401676) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 09:21:20 (1747401680) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 09:21:24 (1747401684) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 09:21:29 (1747401689) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 09:21:33 (1747401693) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 09:21:38 (1747401698) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 09:21:42 (1747401702) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 09:21:47 (1747401707) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (0s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 09:21:47 (1747401707) PASS 45a (29s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 09:22:16 (1747401736) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 09:22:20 (1747401740) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 09:22:25 (1747401745) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 09:22:29 (1747401749) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 09:22:34 (1747401754) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 09:22:38 (1747401758) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 09:22:42 (1747401762) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 09:22:46 (1747401766) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 09:22:47 (1747401767) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (361s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 09:28:48 (1747402128) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 09:28:52 (1747402132) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 09:28:57 (1747402137) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 09:29:01 (1747402141) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 09:29:06 (1747402146) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 09:29:10 (1747402150) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 09:29:14 (1747402154) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 09:29:19 (1747402159) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 09:29:23 (1747402163) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 09:29:23 (1747402163) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 09:29:24 (1747402164) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 09:29:25 (1747402165) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 09:29:26 (1747402166) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 09:29:26 (1747402166) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 09:29:27 (1747402167) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 09:29:28 (1747402168) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (0s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 09:29:28 (1747402168) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00369148 s, 2.8 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 09:29:35 (1747402175) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00373462 s, 42.6 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24337) - No such process PASS 51a (3s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 09:29:38 (1747402178) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00228936 s, 447 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104926 s, 99.9 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 09:29:52 (1747402192) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00554462 s, 185 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 09:29:58 (1747402198) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933754 s, 112 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 09:30:00 (1747402200) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2438 0x986 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2438 0x986 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 09:30:04 (1747402204) llite.lustre-ffff8800b4141000.xattr_cache=1 llite.lustre-ffff88012b32d000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:30:04.000000000 -0400 Modify: 2025-05-16 09:30:04.000000000 -0400 Change: 2025-05-16 09:30:04.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:30:09.000000000 -0400 Modify: 2025-05-16 09:30:09.000000000 -0400 Change: 2025-05-16 09:30:09.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:30:15.000000000 -0400 Modify: 2025-05-16 09:30:15.000000000 -0400 Change: 2025-05-16 09:30:15.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:30:15.000000000 -0400 Modify: 2025-05-16 09:30:15.000000000 -0400 Change: 2025-05-16 09:30:15.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:30:20.000000000 -0400 Modify: 2025-05-16 09:30:20.000000000 -0400 Change: 2025-05-16 09:30:20.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:30:20.000000000 -0400 Modify: 2025-05-16 09:30:20.000000000 -0400 Change: 2025-05-16 09:30:20.000000000 -0400 Birth: - PASS 54 (22s) == sanityn test 55a: rename vs unlink target dir ========= 09:30:26 (1747402226) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 09:30:33 (1747402233) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 09:30:40 (1747402240) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 09:30:52 (1747402252) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 09:31:01 (1747402261) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 09:31:02 (1747402262) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 09:31:04 (1747402264) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 09:31:06 (1747402266) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00266579 s, 24.6 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (0s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 09:31:07 (1747402267) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00317042 s, 20.7 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (0s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 09:31:07 (1747402267) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 09:31:08 (1747402268) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00359556 s, 18.2 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 09:31:09 (1747402269) llite.lustre-ffff8800b4141000.xattr_cache=1 llite.lustre-ffff88012b32d000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 09:31:11 (1747402271) llite.lustre-ffff8800b4141000.xattr_cache=1 llite.lustre-ffff88012b32d000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 09:31:12 (1747402272) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00227401 s, 450 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00199275 s, 514 kB/s 2992: get lock2 2993: get lock1 2992: done 2992: get lock1 2993: done 2993: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 2993: cannot get lock2: Resource deadlock avoided 2993: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 2992: cannot get lock1: Resource deadlock avoided 2992: done 2992: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 09:31:18 (1747402278) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.013745 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844985 s, 124 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3629 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 09:31:25 (1747402285) opening files: ........ closing files: ........ PASS 76 (14s) == sanityn test 77a: check FIFO NRS policy =============== 09:31:39 (1747402299) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0888294 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836074 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112328 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223875 s, 46.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108688 s, 96.5 MB/s , 0.023208 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219892 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0300999 s, 34.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0303014 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291694 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289697 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291436 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324816 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350112 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344639 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337167 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369909 s, 28.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00871382 s, 120 MB/s 1048576 bytes (1.0 MB) copied, 0.00703129 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00904549 s, 116 MB/s 1048576 bytes (1.0 MB) copied, 0.0092551 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121191 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150238 s, 698 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845676 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847037 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00677466 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122699 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00184341 s, 569 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0104657 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.0103108 s, 102 MB/s , 0.011154 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108533 s, 96.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0116483 s, 90.0 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 09:31:41 (1747402301) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0980041 s, 171 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00838282 s, 125 MB/s , 0.00841152 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184419 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228834 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301165 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306741 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254244 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261355 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271316 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227686 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163412 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168386 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0222953 s, 47.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0250162 s, 41.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0208362 s, 50.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277373 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00803804 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00692566 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974372 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825486 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127174 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860012 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00516102 s, 203 MB/s , 0.00178468 s, 588 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992368 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011821 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108178 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100996 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118589 s, 88.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0139269 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121809 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015946 s, 65.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106946 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103641 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222628 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164021 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247138 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269249 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239983 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338303 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165702 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234758 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210204 s, 49.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452719 s, 23.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288974 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450191 s, 23.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505719 s, 20.7 MB/s , 0.0380697 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362991 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086101 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00152475 s, 688 MB/s , 0.00158536 s, 661 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137545 s, 762 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834621 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842634 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0014146 s, 741 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091618 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127623 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137282 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105128 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147194 s, 712 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0035613 s, 294 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105819 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108854 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00321579 s, 326 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=3 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 09:31:44 (1747402304) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.113868 s, 147 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953573 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.0133101 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170849 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131215 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019209 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174866 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025379 s, 41.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312195 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032241 s, 32.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0105119 s, 99.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118198 s, 88.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0249202 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265818 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279087 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333558 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104271 s, 101 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00564199 s, 186 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00460306 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869601 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00316572 s, 331 MB/s 1048576 bytes (1.0 MB) copied, 0.0113107 s, 92.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00865368 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107168 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00537341 s, 195 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00149588 s, 701 MB/s , 0.00152463 s, 688 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011136 s, 94.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00171925 s, 610 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00166013 s, 632 MB/s 1048576 bytes (1.0 MB) copied, 0.00174757 s, 600 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0042983 s, 244 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00314133 s, 334 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0931516 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894462 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201373 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018076 s, 58.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227394 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260392 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272894 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281034 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175339 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240675 s, 43.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293179 s, 35.8 MB/s 1048576 bytes (1.0 MB) copied, 0.037701 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0386022 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365358 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245909 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317243 s, 33.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197053 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120404 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940516 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868682 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086662 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884248 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856717 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944934 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120304 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117206 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125389 s, 83.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129891 s, 80.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143449 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136031 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139802 s, 75.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0149649 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138078 s, 75.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.129487 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00638069 s, 164 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169929 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168573 s, 62.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0203823 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161626 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267612 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275758 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199471 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371833 s, 28.2 MB/s , 0.0267016 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263703 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280137 s, 37.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244728 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342279 s, 30.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0314491 s, 33.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139916 s, 74.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768096 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0071422 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108951 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00184287 s, 569 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0017821 s, 588 MB/s , 0.00553645 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010541 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00160064 s, 655 MB/s 1048576 bytes (1.0 MB) copied, 0.00465718 s, 225 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120637 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012059 s, 87.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135467 s, 77.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 1048576 bytes (1.0 MB) copied, 0.0119678 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143656 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00180676 s, 580 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156146 s, 672 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (5s) == sanityn test 77d: check TRR nrs policy ================ 09:31:49 (1747402309) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.123982 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119535 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158873 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254833 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240724 s, 43.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360618 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376391 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394745 s, 26.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0365869 s, 28.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346179 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434611 s, 24.1 MB/s , 0.0399851 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469073 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438282 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412987 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488713 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051286 s, 20.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112443 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117711 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673264 s, 156 MB/s , 0.00774487 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848283 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165156 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968513 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937897 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968665 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112331 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109714 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116214 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116955 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128796 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138548 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136504 s, 76.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.100008 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709159 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112253 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218666 s, 48.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239778 s, 43.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0261779 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214178 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352434 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283217 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301558 s, 34.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0257531 s, 40.7 MB/s , 0.018252 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157129 s, 66.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0389419 s, 26.9 MB/s , 0.0362675 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234345 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166422 s, 63.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103483 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929993 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0026556 s, 395 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166602 s, 629 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00453231 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916731 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787964 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852734 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115107 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975537 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011381 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152244 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937134 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00667772 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013062 s, 80.3 MB/s , 0.00154648 s, 678 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.108013 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983084 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025653 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232855 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260275 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289584 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258447 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186101 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228751 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237095 s, 44.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244769 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427757 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365747 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534211 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497306 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387359 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447363 s, 23.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00519205 s, 202 MB/s , 0.00417719 s, 251 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00516707 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00644288 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00153593 s, 683 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701304 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145974 s, 71.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00226948 s, 462 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00853962 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118382 s, 88.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100953 s, 104 MB/s , 0.0101072 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103568 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127585 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122138 s, 85.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142837 s, 73.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 09:31:52 (1747402312) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.29@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.156862 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885811 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115729 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178646 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220527 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268161 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261196 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190083 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230194 s, 45.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0288905 s, 36.3 MB/s , 0.0239803 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243528 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155749 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141874 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247461 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00790525 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991531 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114726 s, 91.4 MB/s , 0.0114935 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136635 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140535 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116858 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130103 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012838 s, 81.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0125762 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131838 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0138902 s, 75.5 MB/s 1048576 bytes (1.0 MB) copied, 0.013923 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140563 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136787 s, 76.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.013923 s, 75.3 MB/s , 0.014166 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160059 s, 65.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.102217 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101563 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141999 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233289 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217101 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236629 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937627 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244723 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179329 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178228 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120465 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117907 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306784 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267067 s, 39.3 MB/s , 0.0415329 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283754 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411606 s, 25.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893163 s, 117 MB/s , 0.011068 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106021 s, 98.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00991721 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848706 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100473 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110136 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108323 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977842 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100526 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103751 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113082 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115939 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109011 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110917 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110251 s, 95.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.121542 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103647 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173915 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203279 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227487 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148739 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211921 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020071 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279929 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356439 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199536 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019125 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272203 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208937 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311988 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486709 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313487 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0021725 s, 483 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0102173 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0120057 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119797 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122884 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983451 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129304 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119346 s, 87.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127586 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124735 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148686 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012937 s, 81.1 MB/s , 0.0128069 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012792 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143477 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137541 s, 76.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0935867 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112735 s, 93.0 MB/s , 0.0116706 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274049 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197041 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198035 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246165 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357464 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252627 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340679 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329155 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375084 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039347 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420632 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338376 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487842 s, 21.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396014 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772553 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846091 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961191 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0123881 s, 84.6 MB/s 1048576 bytes (1.0 MB) copied, 0.00915071 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101812 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00338469 s, 310 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845438 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111599 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00303776 s, 345 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00739459 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00151067 s, 694 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00995776 s, 105 MB/s , 0.00700689 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166492 s, 630 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00195752 s, 536 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (7s) == sanityn test 77f: check TBF JobID nrs policy ========== 09:31:59 (1747402319) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.133596 s, 126 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903083 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120118 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950129 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307249 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034645 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147517 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035825 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161448 s, 64.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347502 s, 30.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0341411 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370472 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352178 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301022 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430584 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206742 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291797 s, 35.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00543993 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755508 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117303 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00557622 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939702 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854921 s, 123 MB/s , 0.00969967 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00635475 s, 165 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00864783 s, 121 MB/s 1048576 bytes (1.0 MB) copied, 0.0088319 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964979 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983161 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.0107417 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111666 s, 93.9 MB/s , 0.0110808 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129469 s, 81.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.117298 s, 143 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810135 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895509 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123029 s, 85.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0230292 s, 45.5 MB/s , 0.0254234 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219881 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330431 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211861 s, 49.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230565 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331111 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245192 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372777 s, 28.1 MB/s 1048576 bytes (1.0 MB) copied, 0.034047 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303348 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327129 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386856 s, 27.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00783405 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.006886 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100543 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100093 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840458 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00850356 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122433 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859496 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895184 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119383 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890301 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165124 s, 635 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107523 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108337 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102327 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122022 s, 85.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0972121 s, 173 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102629 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158939 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229098 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167965 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211631 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172902 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205859 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308836 s, 34.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280137 s, 37.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0366334 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272487 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326779 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306091 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027681 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184722 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268451 s, 39.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0024095 s, 435 MB/s , 0.00929984 s, 113 MB/s , 0.00898884 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115006 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098507 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691514 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959794 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119136 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865739 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008383 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995193 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902547 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092029 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117239 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100052 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103067 s, 102 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0948564 s, 177 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128863 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136512 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154279 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190356 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300654 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318769 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328492 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235435 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437475 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340293 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426348 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0555107 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.053933 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528636 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0575661 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0551493 s, 19.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00502863 s, 209 MB/s , 0.00893131 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960119 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0060498 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101687 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109903 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00152796 s, 686 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081432 s, 129 MB/s 1048576 bytes (1.0 MB) copied, 0.00870616 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849177 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147669 s, 710 MB/s 1048576 bytes (1.0 MB) copied, 0.00345011 s, 304 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156484 s, 670 MB/s 1048576 bytes (1.0 MB) copied, 0.00987203 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101418 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115633 s, 90.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 09:32:05 (1747402325) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (2s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 09:32:07 (1747402327) oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg229-client: oleg229-server: ssh exited with exit code 2 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg229-client: oleg229-server: ssh exited with exit code 12 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg229-client: oleg229-server: ssh exited with exit code 12 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg229-client: oleg229-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 09:32:10 (1747402330) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 09:32:16 (1747402336) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0907371 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113034 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123002 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120783 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015712 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164131 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149661 s, 70.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01244 s, 84.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0222377 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014925 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324665 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160571 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188755 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330886 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173604 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300736 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087592 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100364 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983014 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091919 s, 114 MB/s 1048576 bytes (1.0 MB) copied, 0.00986858 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209673 s, 50.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0199644 s, 52.5 MB/s , 0.011302 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204208 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242777 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118226 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141262 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142227 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128308 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140455 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133661 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168977 s, 62.1 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91395 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4304 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (53s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 09:33:09 (1747402389) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.650331 s, 25.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.163508 s, 6.4 MB/s , 0.165029 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169536 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166501 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175862 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174056 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169121 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167267 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.365949 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.371429 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.389269 s, 2.7 MB/s , 0.378565 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.394792 s, 2.7 MB/s 1+0 records in 1+0 records out , 0.386463 s, 2.7 MB/s 1048576 bytes (1.0 MB) copied, 0.386027 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.400258 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144145 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144678 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.147764 s, 7.1 MB/s , 0.144702 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147127 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.150174 s, 7.0 MB/s , 0.145813 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146442 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148462 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.148923 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.148533 s, 7.1 MB/s , 0.149385 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149533 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149178 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.149584 s, 7.0 MB/s 1048576 bytes (1.0 MB) copied, 0.149454 s, 7.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8306 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9643 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.646403 s, 26.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918073 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175002 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103969 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020627 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151179 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01526 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815307 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132286 s, 79.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.367225 s, 2.9 MB/s , 0.3627 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.386479 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378092 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381645 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.371709 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.393906 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388175 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158071 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159059 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156559 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15848 s, 6.6 MB/s 1048576 bytes (1.0 MB) copied, 0.157768 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158912 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158561 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158178 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1584 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158518 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158169 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157988 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159083 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159279 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159559 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161717 s, 6.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8308 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9614 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (131s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 09:35:20 (1747402520) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.189209 s, 88.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199143 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019691 s, 53.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0264496 s, 39.6 MB/s , 0.0194419 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165214 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187853 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331551 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229082 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0707531 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0767293 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0735785 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0762564 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0882278 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0814095 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0956994 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0972354 s, 10.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00556756 s, 188 MB/s , 0.00675936 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714098 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845671 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973128 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106529 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00762184 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848306 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191456 s, 548 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00720132 s, 146 MB/s , 0.00990762 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00130466 s, 804 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948221 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110807 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103726 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129565 s, 80.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93316 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72855 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.29@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.29@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.195088 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835877 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021813 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156193 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253279 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169044 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224574 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00635032 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265356 s, 39.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0675297 s, 15.5 MB/s , 0.066085 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14469 s, 7.2 MB/s , 0.154001 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.266543 s, 3.9 MB/s , 0.261282 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.341001 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36826 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893795 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010293 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00754339 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00754427 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802145 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978255 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926238 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010136 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100131 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108341 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010944 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112087 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122358 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126695 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128451 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133324 s, 78.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93446 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73169 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.29@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.191116 s, 87.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00760711 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00739896 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00770818 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133655 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222668 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144356 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163884 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964091 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0694629 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0749839 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0844201 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0787526 s, 13.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.081272 s, 12.9 MB/s , 0.0802172 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0806097 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0728782 s, 14.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00824028 s, 127 MB/s , 0.00661063 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.006966 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701355 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892496 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890245 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0016354 s, 641 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984047 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00862664 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00349592 s, 300 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087628 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935028 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947674 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956711 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113223 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125992 s, 83.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92918 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87626 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.339496 s, 49.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076762 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00682229 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879117 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204262 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134739 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149673 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085472 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151931 s, 69.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.171614 s, 6.1 MB/s , 0.169161 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.353093 s, 3.0 MB/s , 0.354495 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.372101 s, 2.8 MB/s , 0.367575 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.420812 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.401148 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877316 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00736086 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011642 s, 90.1 MB/s , 0.00807959 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995344 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110731 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012083 s, 86.8 MB/s , 0.0128729 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111563 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165135 s, 635 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109015 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109373 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937316 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103149 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106201 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128872 s, 81.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79206 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72678 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.90955 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72865 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.638239 s, 26.3 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948555 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257629 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152525 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273214 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210531 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273229 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276092 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237258 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372228 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38392 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.379657 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.380308 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.390897 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.396675 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.399383 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.392001 s, 2.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0083572 s, 125 MB/s , 0.00916305 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125214 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107797 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125678 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102618 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128756 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012841 s, 81.7 MB/s , 0.0129232 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130057 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125827 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142549 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123652 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0145 s, 72.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0121591 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013482 s, 77.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8334 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4316 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.218159 s, 76.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268266 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222272 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165628 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289231 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028598 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229174 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0377418 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0408051 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205984 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00765214 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477843 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542088 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473734 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0677703 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0754772 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0807189 s, 13.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0095771 s, 109 MB/s , 0.0102926 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121065 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139222 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012216 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124872 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111092 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00159826 s, 656 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0044555 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132536 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112343 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120447 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118708 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137947 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146165 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170681 s, 61.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93173 s, 21.3 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73864 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.341462 s, 49.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00719891 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211937 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109948 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112826 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255466 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158794 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265448 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241875 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168591 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187234 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178865 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184904 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.315719 s, 3.3 MB/s , 0.322999 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.365717 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.370302 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898509 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998494 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932021 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118417 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914146 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100079 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100751 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119139 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011335 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116789 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012312 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123596 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133935 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124783 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135225 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144867 s, 72.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76697 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73211 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7691 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73627 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (360s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 09:41:21 (1747402881) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 09:41:23 (1747402883) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg229-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.70065 s, 283 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg229-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0156 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 09:42:13 (1747402933) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.122117 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129625 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015864 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293694 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347159 s, 30.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247659 s, 42.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334645 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314305 s, 33.4 MB/s , 0.0340324 s, 30.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442961 s, 23.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0378467 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457307 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349451 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409452 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354407 s, 29.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b4141000.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469786 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401944 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00154153 s, 680 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00130765 s, 802 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881684 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766961 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845327 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.0108519 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985901 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907707 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890803 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00824275 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901485 s, 116 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097622 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123711 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107028 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00138146 s, 759 MB/s 1048576 bytes (1.0 MB) copied, 0.0112527 s, 93.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b4141000.lock_unused_count=2 removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96962 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9765 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.15143 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014835 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294905 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912342 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198199 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290389 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244206 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345118 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263387 s, 39.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0562643 s, 18.6 MB/s , 0.0481438 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151582 s, 6.9 MB/s , 0.148938 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.254136 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.242989 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.341079 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.341279 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249608 s, 42.0 MB/s , 0.0242512 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235858 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247515 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027072 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258967 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253388 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252867 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261172 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257095 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256675 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273884 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266421 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266399 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263626 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248671 s, 42.2 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg229-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94731 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96848 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (68s) == sanityn test 77o: Changing rank should not panic ====== 09:43:21 (1747403001) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 09:43:25 (1747403005) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (28s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 09:43:53 (1747403033) mds.MDS.mdt.nrs_policies=tbf oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg229-client: oleg229-server: ssh exited with exit code 36 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (11s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 09:44:04 (1747403044) oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg229-client: oleg229-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 09:44:07 (1747403047) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:44:08.000000000 -0400 Modify: 2025-05-16 09:44:08.000000000 -0400 Change: 2025-05-16 09:44:08.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 09:44:19 (1747403059) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 09:44:20 (1747403060) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 09:44:21 (1747403061) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 09:44:22 (1747403062) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 09:44:23 (1747403063) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (0s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 09:44:24 (1747403064) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6894 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6894 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 09:44:25 (1747403065) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 09:44:26 (1747403066) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 09:44:33 (1747403073) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (0s) == sanityn test 91: chmod and unlink striped directory === 09:44:33 (1747403073) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 09:44:34 (1747403074) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 09:44:35 (1747403075) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2647 0xa57 0 1 2627 0xa43 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2648 0xa58 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 09:44:43 (1747403083) osc.lustre-OST0000-osc-ffff8800b4141000.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012b32d000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b4141000.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012b32d000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0251936 s, 325 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99877 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4778 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00974062 s, 421 kB/s osc.lustre-OST0000-osc-ffff8800b4141000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012b32d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b4141000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012b32d000.idle_timeout=debug PASS 94 (9s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 09:44:52 (1747403092) llite.lustre-ffff8800b4141000.fast_read=0 llite.lustre-ffff88012b32d000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00153955 s, 8.0 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b4141000.fast_read=1 llite.lustre-ffff88012b32d000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 09:45:00 (1747403100) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 09:45:01 (1747403101) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00123063 s, 3.3 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 09:45:03 (1747403103) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0329112 s, 63.7 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 09:45:05 (1747403105) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0163411 s, 128 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) == sanityn test 100e: DoM: read on open and file size ==== 09:45:06 (1747403106) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00176074 s, 6.8 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 09:45:08 (1747403108) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000529095 s, 7.7 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 09:45:10 (1747403110) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000532137 s, 7.7 MB/s PASS 101b (1s) == sanityn test 101c: Discard DoM data on close-unlink === 09:45:11 (1747403111) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00039758 s, 10.3 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 09:45:15 (1747403115) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1747403115 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1747403118 Finishing test test23 at 1747403118 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 09:45:21 (1747403121) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 09:45:22 (1747403122) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 09:46:09 (1747403169) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 09:46:10 (1747403170) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 09:46:11 (1747403171) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 09:46:12 (1747403172) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0604206 s, 174 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 09:46:16 (1747403176) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0626713 s, 167 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2654 0xa5e 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6894 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (7s) == sanityn test 108a: lseek: parallel updates ============ 09:46:24 (1747403184) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00182382 s, 5.5 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.19522 s, 2.0 MB/s PASS 108a (5s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 09:46:29 (1747403189) 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 3 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 12 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 20 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 23 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg229-server: libkmod: kmod_module_get_size: failed to read coresize from /sys/module/intel_rapl Iteration 24 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 26 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 29 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/intel_rapl/refcnt': 'No such device' oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 41 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 42 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions oleg229-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 45 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_size: failed to read coresize from /sys/module/acpi_cpufreq ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 PASS 109 (389s) == sanityn test 110: do not grant another lock on resend ========================================================== 09:52:58 (1747403578) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:52:59.000000000 -0400 Modify: 2025-05-16 09:52:59.000000000 -0400 Change: 2025-05-16 09:52:59.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg229-client.virtnet oleg229-client.virtnet PASS 110 (49s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 09:53:47 (1747403627) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 09:53:48 (1747403628) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 09:53:49 (1747403629) lustre-OST0000-osc-ffff8800abc5c000 active. lustre-OST0001-osc-ffff8800abc5c000 active. lustre-OST0000-osc-ffff8800abc5c000 active. lustre-OST0001-osc-ffff8800abc5c000 active. MGC192.168.202.129@tcp active. PASS 113 (1s) cleanup: ====================================================== == sanityn test complete, duration 4127 sec ============== 09:53:51 (1747403631) Stopping clients: oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 opts: